Apparatus and method for encoding moving picture by transforming prediction error signal in selected color space, and non-transitory computer-readable storage medium storing program that when executed performs method

ABSTRACT

An apparatus for encoding a moving picture determines either an RGB format or a YUV format, in which intra prediction and inter prediction are executed based on a degree of deviation of information regarding each of R, G, and B components in moving picture data with the RGB format when a prediction selecting one component among three components in a color space format is designated in the input moving picture data with the RGB format, selects either color space format between the RGB format and the YUV format based on a determination result of the determination process, and executes orthogonal transform and quantization on the prediction error signal in the selected color space format and generating an encoded bit stream using a value subjected to the orthogonal transform and the quantization.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority of theprior Japanese Patent Application No. 2015-227686, filed on Nov. 20,2015, the entire contents of which are incorporated herein by reference.

FIELD

The embodiments discussed herein are related to an apparatus forencoding a moving picture, a method of encoding a moving picture, and anon-transitory computer-readable storage medium.

BACKGROUND

Past encoding for moving pictures is mainly executed in the YUV format.The YUV format is a color space format that is formed by a luminancecomponent Y and two color difference components U and V. That is, inpast encoding for moving pictures, input image data is transformed intothe YUV format to be encoded in a case in which the input image data hasthe RGB format. Incidentally, requests for desiring to execute encodingimage signals from imaging apparatuses (input sources) such as digitalcameras while maintaining color space formats of the input sources haverecently increased. Therefore, in the H.265/HEVC standard which is alatest moving picture encoding standard, a mode in which input movingpicture data is encoded while maintaining the RGB format in a case inwhich the input moving picture data has the RGB format has been added.In this encoding mode, a signal with the RGB format is encoded inaccordance with the same scheme as a signal with the YUV format.

This encoding method is realized with screen content coding (SCC) whichis an extension of HEVC. In encoding for a moving picture to which theSCC is applied, in a case in which a moving picture with the RGB formatis input, the moving picture is encoded assuming that a G component(green component) corresponds to the Y component of the YUV format.

In the SCC, a technology called adaptive color transform (ACT) isdefined as an encoding tool. In the ACT, a path through which the RGBformat is transformed into the YUV format is added before a predictionerror signal is subjected to orthogonal transform. Then, either the RGBformat or the YUV format is determined to execute orthogonal transformand quantization for each subblock set in a processing block. In a casein which the ACT is turned on, a format is transformed into the YUVformat and orthogonal transform and quantization are executed. In a casein which the ACT is turned off, orthogonal transform and quantizationare executed while maintaining the RGB format. In the ACT, a paththrough which a subblock with the YUV format in a prediction errorsignal restored by executing inverse quantization and inverse orthogonaltransform on a quantized value (transform coefficient) is transformedinto the RGB format is added, and a decoded image in which all of thesubblocks have the RGB format is generated. In the HEVC standard, aprocessing block is called a coding unit (CU) and a subblcok inorthogonal transform and quantization is called a transform unit (TU).For each subblock included in the processing block, transform betweenthe RGB format and the YUV format is defined as complete reversibletransform so that a problem does not occur even when the ACT is switchedto be turned on and off.

As an example of a technology of the related art, there is known “HEVCScreen Content Coding Extension (SCC)”, [online], [searched on Oct. 7,2015], the Internet <URL: https://hevc.hhi.fraunhofer.de/scc>.

SUMMARY

According to an aspect of the invention, an apparatus for encoding amoving picture includes a memory; and a processor coupled to the memoryand configured to execute a determination process, a selection process,and a transform process. The determination process includes determiningeither color space format between an RGB format and a YUV format, inwhich intra prediction and inter prediction are executed based on adegree of deviation of information regarding each of R, G, and Bcomponents in moving picture data with the RGB format in a case in whicha prediction selecting one component among three components in a colorspace format is designated in the input moving picture data with the RGBformat. The selection process includes selecting either color spaceformat between the RGB format and the YUV format, in which a predictionerror signal for the moving picture data with the RGB format is encodedbased on a determination result of the determination process. Thetransform process includes executing orthogonal transform andquantization on the prediction error signal in the selected color spaceformat and generating an encoded bit stream using a value subjected tothe orthogonal transform and the quantization.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations particularly pointed out inthe claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a functional configuration of a movingpicture encoding apparatus to which ACT is applied;

FIG. 2 is a diagram illustrating a functional configuration of a movingpicture encoding apparatus according to a first embodiment;

FIG. 3 is a flowchart illustrating an encoding process for one pictureaccording to the first embodiment;

FIG. 4 is a flowchart illustrating an encoding process for a picturewith the RGB format;

FIG. 5 is a flowchart illustrating a method of determining whether mACTis turned on or off;

FIG. 6 is a diagram illustrating a functional configuration of a movingpicture encoding apparatus according to a second embodiment;

FIG. 7A is a flowchart (part 1) illustrating an encoding processaccording to the second embodiment;

FIG. 7B is a flowchart (part 2) illustrating the encoding processaccording to the second embodiment;

FIG. 8 is a flowchart illustrating an ACT map generation process;

FIG. 9 is a flowchart illustrating a method of determining whether themACT is turned on or off based on an ACT map;

FIG. 10 is a diagram illustrating a first example of a method ofcalculating an ACT ON ratio;

FIG. 11 is a diagram illustrating a second example of the method ofcalculating the ACT ON ratio;

FIG. 12 is a diagram illustrating a third example of the method ofcalculating the ACT ON ratio;

FIG. 13 is a diagram illustrating a fourth example of the method ofcalculating the ACT ON ratio;

FIG. 14 is a diagram illustrating a functional configuration of a movingpicture encoding apparatus according to a third embodiment;

FIG. 15A is a flowchart (part 1) illustrating an encoding processaccording to the third embodiment;

FIG. 15B is a flowchart (part 2) illustrating the encoding processaccording to the third embodiment; and

FIG. 16 is a diagram illustrating a hardware configuration of acomputer.

DESCRIPTION OF EMBODIMENTS

In encoding for a moving picture, a prediction mode in which threecomponents indicating color space information is independently predictedin intra prediction and inter prediction and a prediction mode in whichany one component of the three components is mainly used for theprediction are prepared.

In a case in which moving picture data with the RGB format is input,prediction using only the R component, prediction using only the Gcomponent, and prediction using only the B component are executed in theprediction mode in which the three components are independentlypredicted. In contrast, in a case in which moving picture data with theRGB format is input, only prediction using, for example, the G componentis executed in the prediction mode in which any one of the threecomponents is used for the prediction.

However, in the prediction mode in which three components, R, G, and Bcomponents, are independently predicted, a calculation amount isconsiderable when a search range of a motion vector in the interprediction is enlarged.

In contrast, in the prediction mode in which any one of the threecomponents, R, G, and B components, is mainly used, there is apossibility of search precision of a motion vector in the interprediction being lowered in a case in which information meaningful inthe selected one component is not concentrated.

As one aspect of the embodiment, provided are solutions for being ableto improve prediction precision when a moving picture with the RGBformat is encoded.

FIG. 1 is a diagram illustrating a functional configuration of a movingpicture encoding apparatus to which ACT is applied.

As illustrated in FIG. 1, a moving picture encoding apparatus 1 to whichthe ACT is applied includes an intra prediction unit 101, an interprediction unit 102, a determination unit 103, a predicted imagegeneration unit 104, and a prediction error signal generation unit 105.The moving picture encoding apparatus 1 includes a color space selectionunit 106, a T/Q processing unit 107, and an ENT processing unit 108. Themoving picture encoding apparatus 1 further includes an IQ/IT processingunit 109, a color space restoration unit 110, a decoded image generationunit 111, a filter processing unit 112, and a frame memory 113.

The intra prediction unit 101 executes intra prediction (also referredto as intra frame prediction) referring to a decoded image obtained bydecoding pixel data encoded in a picture (moving picture data) of acurrent encoding process target. The inter prediction unit 102 executesinter prediction (inter frame prediction) referring to a decoded imageof an encoded picture different from the picture of the current encodingprocess target. The determination unit 103 determines a predictionresult used to generate a predicted image of each processing block basedon a prediction result of the intra prediction unit 101 and a predictionresult of the inter prediction unit 102. The decoded images used for theintra prediction and the inter prediction are stored in the frame memory113. In a case in which a moving picture is encoded in conformity withthe H.265/HEVC standard, intra prediction, inter prediction, anddetermination of prediction results are executed based on a predictionunit (PU) unit set in a processing block (CU).

The predicted image generation unit 104 generates a predicted imagebased on the determination result of the determination unit 103. Theprediction error signal generation unit 105 generates a differencesignal between input data (original image data) and a predicted image ofa processing block in a frame of the current encoding process target asa prediction error signal.

In a case in which moving picture data (frame) with the RGB format isinput, the moving picture encoding apparatus 1 illustrated in FIG. 1executes prediction of an intra frame, prediction of an inter frame,generation of a predicted image, and generation of a prediction errorsignal while maintaining the RGB format.

The color space selection unit 106 selects whether a color space istransformed from the RGB format into the YUV format based on a transformunit (TU) unit set in the processing block. The color space selectionunit 106 includes an ACT unit 106 a and a switch 106 b. In a case inwhich the color space is transformed into the YUV format, the colorspace selection unit 106 switches the switch 106 b to the side of theACT unit 106 a. The ACT unit 106 a transforms the color space of asubblock (TU) from the RGB format into the YUV format using apredetermined transform matrix. A color space format for which completereversible transform with the RGB format is possible, for example, theYCgCo format, is used as the YUV format.

The T/Q processing unit 107 executes orthogonal transform andquantization on the prediction error signal based on the subblock (TU)unit.

The ENT processing unit 108 executes arithmetic encoding such ascontext-based adaptive binary arithmetic coding (CABAC) or anotherentropy encoding process on a value (transform coefficient) quantized bythe T/Q processing unit 107 to generate an encoded bit stream.

The IQ/IT processing unit 109 executes inverse quantization and aninverse orthogonal transform on a value quantized by the T/Q processingunit 107 based on the subblock (TU) unit to restore a prediction errorimage before quantization.

In a case in which a subblock with the YUV format is included in theprediction error image restored by the IQ/IT processing unit 109, thecolor space restoration unit 110 transforms the color space format ofthe subblock into the RGB format. The color space restoration unit 110includes an TACT unit 110 a and a switch 110 b. The color spacerestoration unit 110 specifies a subblock subjected to the orthogonaltransform and the quantization in the YUV format based on transforminformation of a color space of each subblock in the color spaceselection unit 106. In a case in which the YUV format is transformedinto the RGB format, the color space restoration unit 110 switches theswitch 110 b to the side of the IACT unit 110 a. The IACT unit 110 atransforms the color space of the subblock with the YUV (YCgCo) formatinto the RGB format using the predetermined transform matrix.

The decoded image generation unit 111 generates a local decoded imagewith the RGB format in regard to the original image data based on theprediction error image with the RGB format restored by the IQ/ITprocessing unit 109 and the color space restoration unit 110 and thepredicted image with the RGB format and generated by the predicted imagegeneration unit 104.

The filter processing unit 112 executes a filter process such as adeblocking filter process or a sample adaptive offset (SAO) process onthe local decoded image.

The frame memory 113 accumulates the local decoded image subjected tothe filter process. The accumulated local decoded images are used in aprediction process on a frame to be processed subsequently.

As described above, the color space selection unit 106 of the movingpicture encoding apparatus 1 illustrated in FIG. 1 selects either theRGB format or the YUV format to execute the orthogonal transform and thequantization on the prediction error signal. In a case in which theorthogonal transform and the quantization are executed in the YUVformat, the ACT unit 106 a of the color space selection unit 106transforms the prediction error signal from the RGB format into the YUVformat. At this time, the ACT unit 106 a transforms the prediction errorsignal in the YCgCo format for which the complete reversible transformwith the RGB format is possible.

The YCgCo format is an expression format of a color space in which acolor of an image is expressed by a luminance signal Y, a colordifference signal Cg of a green component, and a color difference signalCo of an orange component. A relation indicated in the followingEquation (1) is established between the components Y, Cg, and Co of theYCgCo format and the components R, G, and B of the RGB format.

$\begin{matrix}\begin{matrix}{\begin{bmatrix}G \\B \\R\end{bmatrix} = {{{\frac{1}{8}\begin{bmatrix}1 & 1 & 0 \\1 & {- 1} & {- 1} \\1 & {- 1} & 1\end{bmatrix}}\begin{bmatrix}8 & 0 & 0 \\\beta_{C_{g}} & 8 & 0 \\\beta_{Co} & 0 & 8\end{bmatrix}}\begin{bmatrix}Y \\C_{g} \\C_{o}\end{bmatrix}}} \\{= {{\frac{1}{8}\begin{bmatrix}{8 + \beta_{C_{g}}} & 0 & 0 \\{8 - \left( {\beta_{C_{g}} + \beta_{Co}} \right)} & {- 8} & {- 8} \\{8 - \left( {\beta_{C_{g}} - \beta_{Co}} \right)} & {- 8} & 8\end{bmatrix}}\begin{bmatrix}Y \\C_{g} \\C_{o}\end{bmatrix}}}\end{matrix} & (1)\end{matrix}$

In Equation (1), βCg and βCo are integers.

That is, the ACT unit 106 a of the color space selection unit 106calculates values of the components Y, Cg, and Co from values of thecomponents R, G, and B in the prediction error signal using adeterminant of the matrix inversely transforming Equation (1).

The TACT unit 110 a of the color space restoration unit 110 in themoving picture encoding apparatus 1 in FIG. 1 transforms the restoredprediction error image obtained by executing a T/Q process and an IQ/ITprocess in the YUV (YCgCo) format into the RGB format. That is, the IACTunit 110 a of the color space restoration unit 110 calculates values ofthe components R, G, and B from the values of the components Y, Cg, andCo in the restored prediction error signal using Equation (1).

The moving picture encoding apparatus 1 can encode a moving picture inconformity with the H.265/HEVC standard, as described above, andsupports encoding of a moving picture input with the RGB format byscreen content coding (SCC).

In such a kind of moving picture encoding apparatus 1, for example, amethod of independently encoding three components, the R, G, and Bcomponents can be selected as an encoding method corresponding to a4:4:4 format of the YUV format in a case in which input moving picturedata has the RGB format. In this case, the moving picture encodingapparatus 1 executes the intra frame prediction and the inter frameprediction on each of the three components and executes encoding(monochrome encoding) on each component.

In a case in which the three components are independently encoded withthe foregoing 4:4:4 format, information of the Cg and Co components issmall in the YCgCo format despite the fact that the numbers of pixels ofthe Cg and Co components are the same as the number of pixels of the Ycomponent. Therefore, prediction precision of the Cg and Co componentsdeteriorates. Accordingly, in the case in which the three components areindependently encoded, it is meaningful to execute the prediction in theRGB format rather than the YCgCo format.

However, when moving picture data with the RGB format input to themoving picture encoding apparatus 1 is encoded, it is possible toexecute prediction using any one component among the R, G, and Bcomponents. In this way, when the prediction is executed using any onecomponent, there is a possibility of an adverse influence given to aprediction result in the RGB format due to a difference in signalcharacteristics between the YCgCo format and the RGB format. That is, ina case in which the prediction is executed with the Y component whereinformation of a video is concentrated and a case in which theprediction is executed with the G component among the R, G, and Bcomponents where information of a video is equally distributed,prediction using the Y component having more information is more precisethan prediction using the G component.

Whether the intra frame prediction and the inter frame prediction areexecuted independently on each of three components of a color space orexecuted mainly using one component among the three components isdesignated by a flag “separate_colour_plane_flag” present in a sequenceparameter set (SPS) header. In a case of “separate_colour_plane_flag=1”,the moving picture encoding apparatus 1 executes the intra frameprediction and the inter frame prediction independently on each of thethree components of the color space. Conversely, in a case of““separate_colour_plane_flag=0”, the moving picture encoding apparatus 1executes the intra frame prediction and the inter frame predictionmainly using any one component (for example, the G component) among thethree components of the color space. In other words, the intra frameprediction and the inter frame prediction in the case of“separate_colour_plane_flag=0” are prediction in which the threecomponents in the color space format are not independently predicted andwhich is executed focusing on any one component (for example, the Gcomponent) among the three components of the color space. In thefollowing description, the intra frame prediction and the inter frameprediction in the case of “separate_colour_plane_flag=0” are alsoexpressed as prediction which is predicted using any one component amongthe three components of the color space in some cases.

In this way, in a case in which moving picture data is input with theRGB format and any one component is used among three components R, G,and B for prediction, there is a possibility of precision of a motionvector in the inter prediction being lowered. For example, in a case inwhich each of the three components R, G, and B has meaningfulinformation, the meaningful information of the other two components isnot usable for prediction when the prediction is executed using only anyone component. Therefore, there is a possibility of the predictionprecision deteriorating.

Conversely, when prediction is executed using not only one component(for example, the G component) but also all of the components R, G, andB, the prediction precision becomes better. However, another problemoccurs in that a calculation processing amount increases according tothe number of pixels to be used for the prediction. Specifically, whenprediction is executed using three components, a processing amountincreases three times at the time of a 4:4:4 format, increases twice atthe time of a 4:2:2 format, and increases 1.25 times at the time of a4:2:0 format, compared to a processing amount of a case in whichprediction is executed using one component.

FIG. 2 is a diagram illustrating a functional configuration of a movingpicture encoding apparatus according to a first embodiment.

As illustrated in FIG. 2, a moving picture encoding apparatus 1according to the present embodiment includes an intra prediction unit101, an inter prediction unit 102, a determination unit 103, a predictedimage generation unit 104, and a prediction error signal generation unit105. The moving picture encoding apparatus 1 includes a first colorspace selection unit 106, a T/Q processing unit 107, and an ENTprocessing unit 108. The moving picture encoding apparatus 1 furtherincludes an IQ/IT processing unit 109, a color space restoration unit110, a decoded image generation unit 111, a filter processing unit 112,and a frame memory 113. The moving picture encoding apparatus 1according to the present embodiment further includes an mACTdetermination unit 121, a second color space selection unit 122, and athird color space selection unit 123.

The intra prediction unit 101, the inter prediction unit 102, thedetermination unit 103, the predicted image generation unit 104, and theprediction error signal generation unit 105 in the moving pictureencoding apparatus 1 according to the present embodiment have theforegoing respective functions. The first color space selection unit 106in the moving picture encoding apparatus 1 has the same function as theforegoing first color space selection unit 106 (see FIG. 1). The T/Qprocessing unit 107, the ENT processing unit 108, the IQ/IT processingunit 109, the color space restoration unit 110, the decoded imagegeneration unit 111, the filter processing unit 112, and the framememory 113 in the moving picture encoding apparatus 1 have the foregoingrespective functions.

The mACT determination unit 121 determines which image data with eitherthe RGB format or the YUV format is used for prediction based on, forexample, the degree of deviation in information regarding the RGBcomponents in a current processing target frame. In the followingdescription, in a case in which image data with the RGB format is usedfor prediction of the intra prediction unit 101 and the inter predictionunit 102, mACT is assumed to be turned off. In a case in which imagedata with the YUV format is used in the prediction, the mACT is assumedto be turned on. The mACT determination unit 121 outputs an ON/OFFdetermination result of the mACT to the second color space selectionunit 122 and the third color space selection unit 123. In a case inwhich the mACT is turned on, the mACT determination unit 121 outputs,for example, a signal called “mACT=1” to the second color spaceselection unit 122 and the third color space selection unit 123.Conversely, in a case in which the mACT is turned off, the mACTdetermination unit 121 outputs, for example, a signal called “mACT=0” tothe second color space selection unit 122 and the third color spaceselection unit 123.

Only in a case in which moving picture data input to the moving pictureencoding apparatus 1 has the RGB format, the mACT determination unit 121determines which image with either the RGB format or the YUV format isused for prediction. That is, in a case in which the moving picture datainput to the moving picture encoding apparatus 1 has the YUV format, themACT determination unit 121 determines that an image with the YUV formatis used for prediction (sets the mACT to be turned on).

Based on a determination result of the mACT determination unit 121, thesecond color space selection unit 122 selects a color space of theoriginal image data to be input to the intra prediction unit 101 and theinter prediction unit 102. The second color space selection unit 122includes an ACT unit 122 a and a switch 122 b. The ACT unit 122 atransforms the color space of the original image data from the RGBformat to the YCgCo format. Whether the original image data with the RGBformat input to the second color space selection unit 122 is outputwhile maintaining the RGB format or the original image data with the RGBformat is transformed into the YCgCo format by the ACT unit 122 a isswitched by the switch 122 b. In the case in which the mACT is turnedoff, the second color space selection unit 122 switches the switch 122 bso that the input original image data is output while maintaining theRGB format. Conversely, in the case in which the mACT is turned on, thesecond color space selection unit 122 switches the switch 122 b so thatthe original image data transformed into the YCgCo format by the ACTunit 122 a is output.

Based on the determination result of the mACT determination unit 121,the third color space selection unit 123 selects a color space ofreference image data to be input to the intra prediction unit 101 andthe inter prediction unit 102. The third color space selection unit 123includes an ACT unit 123 a and a switch 123 b. The ACT unit 123 atransforms the color space of the reference image data from the RGBformat into the YCgCo format. Whether the reference image data with theRGB format input to the third color space selection unit 123 is outputwhile maintaining the RGB format or the reference image data with theRGB format is transformed into the YCgCo format by the ACT unit 123 a isswitched by the switch 123 b. In the case in which the mACT is turnedoff, the third color space selection unit 123 switches the switch 123 bso that the input reference image data is output while maintaining theRGB format. Conversely, in the case in which the mACT is turned on, thethird color space selection unit 123 switches the switch 123 b so thatthe reference image data transformed into the YCgCo format by the ACTunit 123 a is output.

Each of the ACT unit 122 a of the second color space selection unit 122and the ACT unit 123 a of the third color space selection unit 123calculates values of the components Y, Cg, and Co from values of thecomponents R, G, and B of the image data by executing inverse transformof Equation (1).

FIG. 3 is a flowchart illustrating an encoding process for one pictureaccording to the first embodiment. FIG. 3 illustrates a flowchart of anencoding process in a case in which a picture with the RGB format isinput and one component is mainly used among three components of a colorspace for prediction. The determination of whether the input picture hasthe RGB format is executed by, for example, an overall control unit (notillustrated in FIG. 2) that controls an overall operation of the movingpicture encoding apparatus 1. The overall control unit determineswhether a picture with the RGB format is encoded base on, for example, aflag “residual_adaptive_colour_transform_enabled_flag” or a tone mappingSEI (header information). Based on a value of the flag“separate_colour_plane_flag”, the overall control unit determineswhether prediction is executed using mainly one component among threecomponents of a color space.

In a case in which the prediction is executed on an input picture withthe RGB format using one component among three components of a colorspace, the moving picture encoding apparatus 1 reads a preferentialprocessing target picture, as illustrated in FIG. 3 (step S1). Then,based on the RGB components of the read picture, it is determinedwhether the mACT is turned on or off (step S2). The steps S1 and S2 areexecuted by the mACT determination unit 121. The mACT determination unit121 outputs a determination result of step S2 to the second color spaceselection unit 122 and the third color space selection unit 123.

Thereafter, the moving picture encoding apparatus 1 executes an encodingprocess (step S3) on the processing target picture. In a case in whichthe prediction is executed on the input picture with the RGB formatmainly using one component among three components of the color space,the moving picture encoding apparatus 1 according to the presentembodiment executes, for example, the same process as a processillustrated in FIG. 4 as the encoding process of step 3.

FIG. 4 is a flowchart illustrating an encoding process for a picturewith the RGB format.

In the encoding process (step S3) on the picture with the RGB format, asillustrated in FIG. 4, it is first determined whether the mACT is turnedon (step S301). The second color space selection unit 122 and the thirdcolor space selection unit 123 execute step S301. In the case in whichthe mACT is turned on (Yes in step S301), the second color spaceselection unit 122 and the third color space selection unit 123 causethe intra prediction unit 101, the inter prediction unit 102, and thedetermination unit 103 to execute prediction using an image with the YUVformat (step S302). That is, in the case in which the mACT is turned on,the second color space selection unit 122 switches the switch 122 b sothat the original image data input to the moving picture encodingapparatus 1 is input to the intra prediction unit 101 and the interprediction unit 102 via the ACT unit 122 a. In the case in which themACT is turned on, the third color space selection unit 123 switches theswitch 123 b so that the reference image data of the frame memory 113 isinput to the intra prediction unit 101 and the inter prediction unit 102via the ACT unit 123 a. Accordingly, the original image data and thereference image data transformed into the YCgCo format are input to eachof the intra prediction unit 101 and the inter prediction unit 102.Thus, each of the intra prediction unit 101 and the inter predictionunit 102 executes prediction using the original image data and thereference image data transformed into the YCgCo format.

In the process of step S302, the intra prediction unit 101 and the interprediction unit 102 execute the prediction based on the prediction block(PU) unit set in the processing block (CU). In a case in which one CU issegmented into a plurality of PUs, each of the intra prediction unit 101and the inter prediction unit 102 executes the prediction on each PU ina Z scan order. When the intra prediction unit 101 and the interprediction unit 102 end the prediction on one processing block, theintra prediction unit 101 and the inter prediction unit 102 output theprediction result to the determination unit 103. The determination unit103 determines a prediction result (predicted image) in which anencoding cost is the minimum based on the prediction results of theintra prediction unit 101 and the inter prediction unit 102. Thedetermination unit 103 outputs the determination result, that is, theprediction result in which the encoding cost is the minimum, to thepredicted image generation unit 104. In this way, the prediction processof step S302 on one processing block (CU) ends. Conversely, in the casein which the mACT is turned off (No in step S301), the second colorspace selection unit 122 and the third color space selection unit 123cause the intra prediction unit 101, the inter prediction unit 102, andthe determination unit 103 to execute the prediction using the imagewith the RGB format (step S303). That is, in the case in which the mACTis turned off, the second color space selection unit 122 switches theswitch 122 b so that the original image data input to the moving pictureencoding apparatus 1 is input to the intra prediction unit 101 and theinter prediction unit 102 without passing through the ACT unit 122 a. Inthe case in which the mACT is turned off, the third color spaceselection unit 123 switches the switch 123 b so that the reference imagedata of the frame memory 113 is input to the intra prediction unit 101and the inter prediction unit 102 without passing through the ACT unit123 a. Accordingly, the original image data and the reference image datawith the RGB format are input to each of the intra prediction unit 101and the inter prediction unit 102. Thus, each of the intra predictionunit 101 and the inter prediction unit 102 executes the prediction usingthe original image data and the reference image data with the RGBformat.

Even in the process of step S303, the intra prediction unit 101 and theinter prediction unit 102 output the prediction result to thedetermination unit 103 when ending the prediction on one processingblock. The determination unit 103 determines a prediction result(predicted image) in which the encoding cost is the minimum based on theprediction results of the intra prediction unit 101 and the interprediction unit 102. The determination unit 103 outputs the predictionresult, that is, the prediction result in which the encoding cost is theminimum, to the predicted image generation unit 104. Accordingly, theprediction process of step S303 on one processing block (CU) ends.

When the prediction process of step S302 or S303 ends, the predictedimage generation unit 104 subsequently generates a predicted image withthe RGB format based on the prediction result (step S304). In the casein which the mACT is turned off, each of the intra prediction unit 101and the inter prediction unit 102 executes the prediction using imagedata with the RGB format. Therefore, in the case in which the mACT isturned off, the predicted image generation unit 104 generates thepredicted image using, for example, the reference image data with theRGB format input from the intra prediction unit 101 or the interprediction unit 102 via the determination unit 103. Conversely, in thecase in which the mACT is turned on, each of the intra prediction unit101 and the inter prediction unit 102 executes the prediction using theimage data with the YCgCo. Therefore, in the case in which the mACT isturned on, the predicted image generation unit 104 reads the image datawith the RGB format corresponding to the prediction result in which theencoding cost is the minimum from the frame memory 113 and generates thepredicted image. The predicted image generation unit 104 outputs thegenerated predicted image to the prediction error signal generation unit105. The predicted image is also used when a local decoded image isgenerated. Therefore, the predicted image generated by the predictedimage generation unit 104 is output to the prediction error signalgeneration unit 105 and is also stored in, for example, a bufferincluded in the decoded image generation unit 111 or the like.

After the predicted image is generated, the prediction error signalgeneration unit 105 in the moving picture encoding apparatus 1 generatesa prediction error signal (step S305). The prediction error signalgeneration unit 105 obtains a difference between the predicted image andthe original image data in regard to the processing block (CU) andoutputs the difference as a prediction error signal to the first colorspace selection unit 106. The prediction error signal generation unit105 generates the prediction error signal with the RGB format using theoriginal image data and the predicted image with the RGB format.Therefore, the prediction error signal with the RGB format is input tothe first color space selection unit 106.

After the prediction error signal is generated, the first color spaceselection unit 106 in the moving picture encoding apparatus 1 executesthe ACT process (step S306). The first color space selection unit 106determines whether the prediction error signal is output to the T/Qprocessing unit 107 while maintaining the RGB format or the predictionerror signal is transformed into the YCgCo format and is output to theT/Q processing unit 107 for each transform block (TU) set in theprocessing block (CU). Which prediction error signal with either the RGBformat or the YCgCo format is output to the T/Q processing unit 107 isdetermined based on, for example, a control signal from the overallcontrol unit (not illustrated in FIG. 2). In a case in which theprediction error signal is output to the T/Q processing unit 107 whilemaintaining the RGB format, the first color space selection unit 106switches the switch 106 b so that the prediction error signal from theprediction error signal generation unit 105 is input to the T/Qprocessing unit 107 without passing through the ACT unit 106 a.Conversely, in a case in which the prediction error signal istransformed into the YCgCo format and is output to the T/Q processingunit 107, the first color space selection unit 106 switches the switch106 b so that the prediction error signal from the prediction errorsignal generation unit 105 is input to the T/Q processing unit 107 viathe ACT unit 106 a. The ACT unit 106 a transforms the prediction errorsignal from the RGB format to the YCgCo format through the inversetransform of Equation (1). When the ACT process on one processing blockends, the first color space selection unit 106 stores color spaceinformation regarding each transform block to, for example, a buffer(not illustrated) included in the color space restoration unit 110 orthe like.

When the prediction error signal is input to the T/Q processing unit 107through the ACT process executed by the first color space selection unit106, the T/Q processing unit 107 executes the orthogonal transform andthe quantization based on the transform block (TU) unit (step S307). TheT/Q processing unit 107 determines whether the transform block haseither the RGB format or the YCgCo format and executes the orthogonaltransform and the quantization corresponding to each format. A value(transform coefficient) quantized by the T/Q processing unit 107 isoutput to the ENT processing unit 108. When the transform coefficientquantized by the T/Q processing unit 107 is input, the ENT processingunit 108 executes an ENT process of executing arithmetic encoding orentropy encoding on the input transform coefficient to generate a bitstream (step S308).

The transform coefficient quantized by the T/Q processing unit 107 isalso used to generate a reference image (local decoded image) at thetime of encoding of a subsequent processing block or picture. That is,after the orthogonal transform and the quantization of step S307 areexecuted, the moving picture encoding apparatus 1 executes the ENTprocess and also executes a decoded image generation process (step S309)and a filter process (step S310). The decoded image generation processof step S309 is executed by the IQ/IT processing unit 109, the colorspace restoration unit 110, and the decoded image generation unit 111.The filter process of step S310 is executed by the filter processingunit 112.

In the decoded image generation process of step S309, the IQ/ITprocessing unit 109 executes the orthogonal transform and thequantization on the transform coefficient quantized by the T/Qprocessing unit 107 to restore the prediction error image before theexecution of the orthogonal transform. Next, the color space restorationunit 110 executes inverse transform on the transform block with theYCgCo format in the RGB format based on ON/OFF information of the ACT inregard to each transform block (TU) in the prediction error image(processing block) to restore the prediction error signal with the RGBformat. Thereafter, the decoded image generation unit 111 generates adecoded image with the RGB format in regard to the original image datausing the prediction error signal restored in the RGB format and thepredicted image generated by the predicted image generation unit 104.

In the filter process of step S310, for example, the filter processingunit 112 executes a deblocking filter process on the decoded imagegenerated by the decoded image generation unit 111. In a case in which amoving picture is encoded in conformity with the H.265/HEVC standard,for example, the filter processing unit 112 continuously executes theSAO process after the deblocking filter process. When the predeterminedfilter process on the decoded image ends, the filter processing unit 112stores the decoded image subjected to the filter process in the framememory 113. Accordingly, the encoding process on one processing blockends in the moving picture encoding apparatus 1.

In the moving picture encoding apparatus 1, the processes of steps S301to S310 are sequentially executed on each of the plurality of processingblocks set in one picture. At this time, the moving picture encodingapparatus 1 executes the processes of steps S301 to S310 on eachprocessing block in a pipeline way.

In this way, in a case in which the picture with the RGB format is inputand the prediction is executed mainly using one component among threecomponents of the color space, which image with either the RGB format orthe YCgCo format is used for the prediction is determined in the processof encoding the moving picture according to the present embodiment.

In the YUV format including the YCgCo format, meaningful information ofthe image data is concentrated on the luminance signal Y. Therefore, forexample, excluding a case in which the meaningful information isconcentrated on one of the R, G, and B components in the RGB format, theprediction can be executed with higher precision when the prediction isexecuted while maintaining the RGB format than when the prediction isexecuted by transforming the RGB format into the YCgCo format.

As described above, it is better to execute the prediction whilemaintaining the RGB format, for example, in a case in which themeaningful information is concentrated on one of the R, G, and Bcomponents in the RGB format. For example, in a case in which themeaningful information in the picture with the RGB format isconcentrated on the G component, information (signal) regarding the Gcomponent disperses to three signals, the luminance signal Y, the colordifference signal Cg of the green component, and the color differencesignal Co of the orange component when the picture is transformed intothe YCgCo format. Therefore, meaningful information included in theluminance signal Y important in the YUV format is small, and thus thereis a low possibility of the prediction precision being lower than in acase in which the prediction is executed while maintaining the RGBformat. Accordingly, in the moving picture encoding apparatus 1according to the present embodiment, the mACT determination unit 121determines which format is used between the RGB format and the YCgCoformat to execute the prediction with higher precision. Hereinafter, amethod of determining which format is used between the RGB format andthe YCgCo format by the mACT determination unit 121 for the prediction,that is, a method of determining whether the mACT is turned on or off instep S2, will be described with reference to FIG. 5.

FIG. 5 is a flowchart illustrating a method of determining whether themACT is turned on or off.

After reading a processing target picture, the mACT determination unit121 first calculates block averages AveOrg_R, AveOrg_G, and AveOrg_B andvariances VarOrg_R, VarOrg_G, and VarOrg_B in regard to the R, G, and Bcomponents, as illustrated in FIG. 5 (step S201). In step S201, the mACTdetermination unit 121 calculates a block average AveOrg_G and avariance VarOrg_G in regard to the G component using, for example, thefollowing Equations (2-1) and (2-2).

$\begin{matrix}{{AveOrg\_ G} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}{{Org\_ G}\lbrack i\rbrack}}}} & \left( {2\text{-}1} \right) \\{{VarOrg\_ G} = {\frac{1}{N}{\sum\limits_{i = 1}^{N}\left( {{{Org\_ G}\lbrack i\rbrack} - {AveOrg\_ G}} \right)}}} & \left( {2\text{-}2} \right)\end{matrix}$

In Equations (2-1) and (2-2), N indicates a total number of pixels in ablock and i indicates a variable indicating a pixel in the block. InEquations (2-1) and (2-2), Org_G[i] indicates a pixel value of the Gcomponent of an i-th pixel in an original image.

In step S201, the mACT determination unit 121 calculates the blockaverage AveOrg_R and the variance VarOrg_R of the R component, the blockaverage AveOrg_B and the variance VarOrg_B of the B component using thesame equations as Equations (2-1) and (2-2).

Next, the mACT determination unit 121 calculates variances PicVarOrg_R,PicVarOrg_G, and PicVarOrg_B in the picture unit in regard to the R, G,and B components (step S202). In step S202, the mACT determination unit121 calculates, for example, a variance PicVarOrg_G in the picture unitin regard to the G component using the following Equation (3).

$\begin{matrix}{{PicVarOrg\_ G} = {\frac{1}{M}{\sum\limits_{j = 1}^{M}{{VarOrg\_ G}\lbrack j\rbrack}}}} & (3)\end{matrix}$

In Equation (3), M indicates the number of blocks in a picture and jindicates a variable representing a block in the picture.

In step S202, the mACT determination unit 121 calculates the variancesPicVarOrg_R and PicVarOrg_B in the picture unit in regard to the R and Bcomponents using the same equation as Equation (3).

Next, the mACT determination unit 121 determines whether the mACT isturned on or off based on the variances PicVarOrg_R, PicVarOrg_G, andPicVarOrg_B in the picture unit in regard to the R, G, and B componentscalculated in step S202 (step S203). In step S203, as illustrated inFIG. 5, for example, using determination threshold values TH_R, TH_G,and TH_B of the R, G, and B components, the mACT determination unit 121determines that the mACT is turned off in a case in which the variancesof two or more components among the variances PicVarOrg_R, PicVarOrg_G,and PicVarOrg_B of three components are less than the determinationthreshold values.

As described above, in a case in which the prediction is executed mainlyusing one component among the three components in the picture input withthe RGB format, the mACT determination unit 121 determines whether theprediction is executed while maintaining the RGB format or theprediction is executed by transforming the RGB format into the YCgCoformat. Excluding a case in which the meaningful information isconcentrated on one of the R, G, and B components in the picture withthe RGB format, the prediction can be executed with higher precisionwhen the prediction is executed while maintaining the RGB format thanwhen the prediction is executed by transforming the RGB format into theYCgCo format.

On the other hand, since the transform of the RGB format into the YCgCoformat is calculation using a 3×3 matrix, transform calculation on onepixel can be realized by multiplication of three times and addition ofthree times. In practice, a calculation amount is increased twice sincethe transform process is executed on the original image data and thetransform process is executed on the reference image data at the time ofthe prediction. Additionally, since the calculation amount is furtherincreased about twice by executing the inverse transform from the YCgCoformat to the RGB format, the calculation amount per one pixel is aboutmultiplication of 12 times and addition of 8 times. This can be said tobe a process in which the calculation amount is slighter than acalculation amount of difference absolute value calculation ordifference square calculation corresponding to the number of searchpoints per pixel in motion search in which several tens of points orseveral thousands of points are searched, or a calculation amount of asub-pel filter process executed at the time of sub-pel search.

Accordingly, in the moving picture encoding according to the presentembodiment, it is possible to execute the prediction with high precisionwhile suppressing an increase in a calculation amount when a picturewith the RGB format is encoded.

The YCgCo format exemplified in the present embodiment is merely anexample of the YUV format for which the complete reversible transformwith the RGB format is possible. The RGB format may be transformed intoanother YUV format as long as complete reversible transform orsubstantial complete reversible transform with the RGB format ispossible.

Whether the mACT according to the present embodiment is determined to beturned on or off may be executed based on another value without beinglimited to the variances in the picture unit of the R, G, and Bcomponents in the processing target picture.

In the present embodiment, the example of the encoding process in thecase in which the picture with the RGB format is input has beendescribed. However, the moving picture encoding apparatus 1 according tothe present embodiment can also encode a picture with the YUV format. Ina case in which a picture with the YUV format is input, the movingpicture encoding apparatus 1 executes an existing encoding process onimage data with the YUV format. At this time, as described above, themACT determination unit 121 determines that the prediction is executedin the YUV format (the mACT is turned on), and the second color spaceselection unit 122 and the third color space selection unit 123 inputthe image data of the YUV format to the intra prediction unit 101 andthe inter prediction unit 102.

FIG. 6 is a diagram illustrating a functional configuration of a movingpicture encoding apparatus according to a second embodiment.

As illustrated in FIG. 6, a moving picture encoding apparatus 1according to the present embodiment includes an intra prediction unit101, an inter prediction unit 102, a determination unit 103, a predictedimage generation unit 104, and a prediction error signal generation unit105. The moving picture encoding apparatus 1 includes a first colorspace selection unit 106, a T/Q processing unit 107, and an ENTprocessing unit 108. The moving picture encoding apparatus 1 furtherincludes an IQ/IT processing unit 109, a color space restoration unit110, a decoded image generation unit 111, a filter processing unit 112,and a frame memory 113. The moving picture encoding apparatus 1according to the present embodiment further includes an mACTdetermination unit 125, a second color space selection unit 122, a thirdcolor space selection unit 123, and an ACT map generation unit 126.

The intra prediction unit 101, the inter prediction unit 102, thedetermination unit 103, the predicted image generation unit 104, and theprediction error signal generation unit 105 in the moving pictureencoding apparatus 1 according to the present embodiment have theforegoing respective functions. The first color space selection unit 106in the moving picture encoding apparatus 1 has the same function as theforegoing first color space selection unit 106 (see FIG. 1). The T/Qprocessing unit 107, the ENT processing unit 108, the IQ/IT processingunit 109, the color space restoration unit 110, the decoded imagegeneration unit 111, the filter processing unit 112, and the framememory 113 in the moving picture encoding apparatus 1 have the foregoingrespective functions. The frame memory 113 stores an ACT map 113 bgenerated by the ACT map generation unit 126 in addition to a decodedimage 113 a decoded from the quantized transform coefficient.

The mACT determination unit 125 of the moving picture encoding apparatus1 according to the present embodiment determines which image with eitherthe RGB format or the YUV (YCgCo) format is used for prediction, inother words, determines whether the ACT (mACT) for prediction is turnedon or off. The mACT determination unit 125 determines whether the mACTis turned on or off based on an ACT map stored in the frame memory 113.The mACT determination unit 125 outputs an ON/OFF determination resultof the mACT to the second color space selection unit 122 and the thirdcolor space selection unit 123. In a case in which the mACT is turnedon, the mACT determination unit 125 outputs, for example, a signalcalled “mACT=1” to the second color space selection unit 122 and thethird color space selection unit 123. Conversely, in a case in which themACT is turned off, the mACT determination unit 125 outputs, forexample, a signal called “mACT=0” to the second color space selectionunit 122 and the third color space selection unit 123.

In a case in which a picture (moving picture data) input to the movingpicture encoding apparatus 1 has the RGB format and the prediction isexecuted mainly using one component among three components, the mACTdetermination unit 125 determines which image data with either the RGBformat or the YUV (YCgCo) format is used for prediction. That is, in acase in which the picture input to the moving picture encoding apparatus1 has the YUV format, the mACT determination unit 125 determines thatthe prediction is executed using the image data with the YUV format(determines that the mACT is turned on). The mACT determination unit 125determines whether the input moving picture data has the RGB format orthe YUV format by a control signal from an overall control unit (notillustrated in FIG. 6) that controls an overall operation of the movingpicture encoding apparatus 1.

The second color space selection unit 122 selects a color space oforiginal image data input to the intra prediction unit 101 and the interprediction unit 102 based on a determination result of the mACTdetermination unit 125. The second color space selection unit 122includes an ACT unit 122 a and a switch (not illustrated). The ACT unit122 a transforms the color space of the original image data from the RGBformat to the YCgCo format. Whether the original image data with the RGBformat input to the second color space selection unit 122 is outputwhile maintaining the RGB format or the original image data with the RGBformat is transformed into the YCgCo format by the ACT unit 122 a isswitched by the switch.

Based on the determination result of the mACT determination unit 125,the third color space selection unit 123 selects a color space ofreference image data to be input to the intra prediction unit 101 andthe inter prediction unit 102. The third color space selection unit 123includes an ACT unit 123 a and a switch (not illustrated). The ACT unit123 a transforms the color space of the reference image data from theRGB format into the YCgCo format. Whether the reference image data withthe RGB format input to the third color space selection unit 123 isoutput while maintaining the RGB format or the reference image data withthe RGB format is transformed into the YCgCo format by the ACT unit 123a is switched by the switch 123 b.

Each of the ACT unit 122 a of the second color space selection unit 122and the ACT unit 123 a of the third color space selection unit 123calculates values of the components Y, Cg, and Co from values of thecomponents R, G, and B of the image data by executing inverse transformof Equation (1).

The ACT map generation unit 126 generates an ACT map including screenposition information and the number of counts by counting whetherencoding (orthogonal transform and quantization) is executed using a oneprocess between ACT ON and ACT OFF for each picture.

FIG. 7A is a flowchart (part 1) illustrating an encoding processaccording to the second embodiment. FIG. 7B is a flowchart (part 2)illustrating the encoding process according to the second embodiment.FIGS. 7A and 7B illustrate flowcharts of an encoding process on oneprocessing block in a case in which the picture (the original imagedata) with the RGB format is input and the prediction is executed mainlyusing one component among three components of the color space. Theprediction executed using mainly one component among three components ofthe color space is prediction of a case in which a flag“separate_colour_plane_flag” present in the SPS header is“separate_colour_plane_flag=0”, as described in the first embodiment.For example, the overall control unit (not illustrated in FIG. 6)controlling an overall operation of the moving picture encodingapparatus 1 determines whether the input picture has the RGB format. Theoverall control unit determines whether the prediction is executedmainly using one component among three components, for example, by avalue of the flag “separate_colour_plane_flag”.

In a case in which the prediction is executed on the input picture withthe RGB format mainly using one component among three components of thecolor space, the moving picture encoding apparatus 1 first determineswhether there is the ACT map usable for the ON/FF determination of themACT, as illustrated in FIG. 7A (step S11). The mACT determination unit125 executes step S11. In a case in which there is the ACT map usablefor the determination (Yes in step S11), the mACT determination unit 125determines whether the mACT is turned on or off based on the ACT map(step S12). Conversely, in a case in which there is no ACT map usablefor the determination (No in step S11), the mACT determination unit 125determines whether the mACT is turned on or off based on initial setting(step S13). The mACT determination unit 125 outputs the determinationresult of step S12 or S13 to the second color space selection unit 122and the third color space selection unit 123. The initial setting in thedetermination of step S13 is, for example, setting in which the mACT isnormally turned on or setting in which the determination is executedusing variances or the like of the RGB components, as in the firstembodiment.

Thereafter, in the moving picture encoding apparatus 1, a predictionprocess is executed based on the ON/OFF determination result of the mACT(step S14). The prediction process of step S14 is executed incooperation with the second color space selection unit 122, the thirdcolor space selection unit 123, the intra prediction unit 101, the interprediction unit 102, and the determination unit 103. The second colorspace selection unit 122 and the third color space selection unit 123switches the switch based on the ON/OFF determination result of themACT.

In a case in which the mACT is turned on, the second color spaceselection unit 122 outputs the original image data transformed into theYCgCo format by the ACT unit 122 a to the intra prediction unit 101 andthe inter prediction unit 102. In the case in which the mACT is turnedon, the third color space selection unit 123 outputs a decoded image(reference image data) read from the frame memory 113 and transformedinto the YCgCo format by the ACT unit 123 a to the intra prediction unit101 and the inter prediction unit 102.

Conversely, in a case in which the mACT is turned off, the second colorspace selection unit 122 outputs the input original image data with theRGB format to the intra prediction unit 101 and the inter predictionunit 102, while maintaining the RGB format. In the case in which themACT is turned on, the third color space selection unit 123 outputs thedecoded image (the reference image data) with the RGB format read fromthe frame memory 113 to the intra prediction unit 101 and the interprediction unit 102, while maintaining the RGB format.

Each of the intra prediction unit 101 and the inter prediction unit 102executes the prediction using the original image data input from thesecond color space selection unit 122 and the reference image data inputfrom the third color space selection unit 123. Each of the intraprediction unit 101 and the inter prediction unit 102 executes theprediction based on the prediction block (PU) unit set in the processingblock (CU). When the intra prediction unit 101 and the inter predictionunit 102 end the prediction on each prediction block included in oneprocessing block, the intra prediction unit 101 and the inter predictionunit 102 output the prediction result to the determination unit 103. Thedetermination unit 103 determines a prediction result (predicted image)in which an encoding cost is the minimum based on the prediction resultsof the intra prediction unit 101 and the inter prediction unit 102. Thedetermination unit 103 outputs the determination result, that is, theprediction result in which the encoding cost is the minimum, to thepredicted image generation unit 104. Accordingly, the prediction processof step S14 on one processing block (CU) ends.

When the process of step S14 ends, the predicted image generation unit104 subsequently generates a predicted image with the RGB format basedon the prediction result (step S15). In the case in which the mACT isturned off, each of the intra prediction unit 101 and the interprediction unit 102 executes the prediction using image data with theRGB format. Therefore, in the case in which the mACT is turned off, thepredicted image generation unit 104 generates the predicted image usingthe reference image data with the RGB format input from the intraprediction unit 101 or the inter prediction unit 102 via thedetermination unit 103. Conversely, in the case in which the mACT isturned on, each of the intra prediction unit 101 and the interprediction unit 102 executes the prediction using the image data withthe YCgCo format. Therefore, in the case in which the mACT is turned on,the predicted image generation unit 104 reads the image data with theRGB format corresponding to the prediction result in which the encodingcost is the minimum from the frame memory 113 and generates thepredicted image. The predicted image generation unit 104 outputs thegenerated predicted image to the prediction error signal generation unit105. The predicted image is also used when a local decoded image isgenerated. Therefore, the predicted image generated by the predictedimage generation unit 104 is output to the prediction error signalgeneration unit 105 and is also stored in, for example, a buffer (notillustrated) included in the decoded image generation unit 111 or thelike.

After the predicted image is generated, the prediction error signalgeneration unit 105 in the moving picture encoding apparatus 1 generatesa prediction error signal (step S16). The prediction error signalgeneration unit 105 obtains a difference between the predicted image andthe original image data in regard to the processing block (CU) andoutputs the difference as a prediction error signal to the first colorspace selection unit 106. The prediction error signal generation unit105 generates the prediction error signal with the RGB format using theoriginal image data and the predicted image with the RGB format.Therefore, the prediction error signal with the RGB format is input tothe first color space selection unit 106.

After the prediction error signal is generated, as illustrated in FIG.7B, the first color space selection unit 106 in the moving pictureencoding apparatus 1 executes the ACT process (step S17). The firstcolor space selection unit 106 determines whether the prediction errorsignal is output to the T/Q processing unit 107 while maintaining theRGB format or the prediction error signal is transformed into the YCgCoformat and is output to the T/Q processing unit 107 for each transformblock (TU) set in the processing block (CU). Which prediction errorsignal with either the RGB format or the YCgCo format is output to theT/Q processing unit 107 is determined based on, for example, a controlsignal from the overall control unit (not illustrated in FIG. 6). In acase in which the prediction error signal is output to the T/Qprocessing unit 107 while maintaining the RGB format, the first colorspace selection unit 106 switches the switch so that the predictionerror signal from the prediction error signal generation unit 105 isinput to the T/Q processing unit 107 without passing through the ACTunit 106 a. Conversely, in a case in which the prediction error signalis transformed into the YCgCo format and is output to the T/Q processingunit 107, the first color space selection unit 106 switches the switchso that the prediction error signal from the prediction error signalgeneration unit 105 is input to the T/Q processing unit 107 via the ACTunit 106 a. The ACT unit 106 a transforms the prediction error signalfrom the RGB format to the YCgCo format through the inverse transform ofEquation (1). When the ACT process on one processing block (CU) ends,the first color space selection unit 106 stores ON/OFF information ofthe ACT regarding each transform block to, for example, a buffer (notillustrated) included in the color space restoration unit 110 or thelike.

When the prediction error signal is input to the T/Q processing unit 107through the ACT process executed by the first color space selection unit106, the T/Q processing unit 107 executes the orthogonal transform andthe quantization based on the transform block (TU) unit (step S18). TheT/Q processing unit 107 determines whether the transform block haseither the RGB format or the YCgCo format and executes the orthogonaltransform and the quantization corresponding to each format. A transformcoefficient quantized by the T/Q processing unit 107 is output to theENT processing unit 108. When the transform coefficient quantized by theT/Q processing unit 107 is input, the ENT processing unit 108 executesan ENT process of executing arithmetic encoding or entropy encoding onthe input transform coefficient to generate a bit stream (step S19).

The transform coefficient quantized by the T/Q processing unit 107 isalso used to generate a reference image (local decoded image) at thetime of encoding of a subsequent processing block or picture. That is,after the orthogonal transform and the quantization of step S18 areexecuted, the moving picture encoding apparatus 1 executes the ENTprocess and also executes a decoded image generation process (step S20)and a filter process (step S21). The decoded image generation process ofstep S20 is executed by the IQ/IT processing unit 109, the color spacerestoration unit 110, and the decoded image generation unit 111. Thefilter process of step S21 is executed by the filter processing unit112. In the decoded image generation process of step S20, the IQ/ITprocessing unit 109 first executes the inverse orthogonal transform andthe inverse quantization on the transform coefficient quantized by theT/Q processing unit 107 to restore the prediction error image before theexecution of the orthogonal transform. Next, the color space restorationunit 110 executes inverse transform on the transform block with theYCgCo format in the RGB format based on ON/OFF information of the ACT inregard to each transform block (TU) in the prediction error image(processing block) to restore the prediction error signal with the RGBformat. Thereafter, the decoded image generation unit 111 generates adecoded image with the RGB format in regard to the original image datausing the prediction error signal restored in the RGB format and thepredicted image generated by the predicted image generation unit 104. Inthe filter process of step S21, for example, the filter processing unit112 executes a filter process on the decoded image generated by thedecoded image generation unit 111. In a case in which a moving pictureis encoded in conformity with the H.265/HEVC standard, for example, thefilter processing unit 112 continuously executes the SAO process afterthe deblocking filter process. When the predetermined filter process onthe decoded image ends, the filter processing unit 112 stores thedecoded image subjected to the filter process in the frame memory 113.

Further, in the moving picture encoding apparatus 1 according to thepresent embodiment generates the decoded image in step S20, andsubsequently executes the ACT map generation process (step S22) alongwith the filter process (step S21). The ACT map generation process ofstep S22 is executed by the ACT map generation unit 126. The ACT mapgeneration unit 126 generates the ACT map including screen positioninformation and the number of counts in regard to the TU subjected tothe orthogonal transform and quantization by turning on the ACT, basedon the ON/OFF information of the ACT for each transform block (TU) usedby the color space restoration unit 110. The ACT map generation unit 126stores the generated ACT map in the frame memory 113. The ACT map storedin the frame memory 113 is referred to by the mACT determination unit125 to determine whether the mACT is turned on or off at the time ofexecution of the subsequent processing block or picture.

As described above, the encoding process on one processing block (CU)ends in the moving picture encoding apparatus 1.

In the moving picture encoding apparatus 1, the processes of steps S11to S22 are sequentially executed on each of the plurality of processingblocks set in one picture. At this time, the moving picture encodingapparatus 1 executes the processes of steps S11 to S22 on eachprocessing block in a pipeline way.

FIG. 8 is a flowchart illustrating an ACT map generation process.

In the ACT map generation process of step S22, as illustrated in FIG. 8,the ACT map generation unit 126 first selects the subblock (TU) (stepS2201) and acquires the ON/OFF information of the ACT at the time ofquantization of the subblock (step S2202). The ACT map generation unit126 acquires, for example, ON/OFF information of IACT in the color spacerestoration unit 110, that is, information indicating whether totransform the selected TU from the YCgCo format to the RGB format, asthe ON/OFF information of the ACT at the time of quantization.

Next, based on the acquired ON/OFF information of the ACT, the ACT mapgeneration unit 126 determines whether the setting of the ACT of theselected subblock is turned on (step S2203). In a case in which thesetting of the ACT is turned on (Yes in step S2203), the ACT mapgeneration unit 126 calculates a count Count_mACT according to the sizeof the subblock (TU) (step S2204). Conversely, in a case in which thesetting of the ACT is turned off (No in step S2203), the ACT mapgeneration unit 126 takes a count Count_mACT=0 in regard to the subblock(TU).

Next, the ACT map generation unit 126 determines whether the process isexecuted on all of the subblocks in the processing block (step S2206).In a case in which there is an unprocessed subblock (No in step S2206),the ACT map generation unit 126 executes the processes of steps S2202 toS2205 on the unprocessed subblock. In a case in which the process isexecuted on all of the subblocks (Yes in step S2206), the ACT mapgeneration unit 126 stores a position ACT_map and a count Count_mACT ofeach subblock as the ACT map in the frame memory 113 (step S2207).Accordingly, the ACT map generation process on one processing blockends.

Next, a method of calculating the count Count_mACT of step S2204 in theACT map generation process will be described.

The size of the transform block (TU) which is the subblock in the ACTmap generation process indicates the number of pixels in the transformblock. Since the TU is set such that recursive quad-tree segmentation isexecuted on the processing block (CU), the number of pixels in the TU isdifferent depending on the size of the CU and a segmentation form. Thatis, an influence of information regarding the pixels in the TU on theentire screen is different depending on the size of the TU. Therefore,when the count Count_mACT is calculated in step S2204, a value accordingto the size of each TU is calculated using a minimum size of the TU as astandard. That is, in a case in which a value (log 2TrafoSize)indicating how many powers of 2 the size of the currently processed TUhas is a value (Log 2MinTrafoSize) indicating a minimum size of the TUdecided separately at the time of encoding, the count Count_mACT is setto +1.

In a case in which the size log 2TrafoSize of the processing target TUis a size greater by one step than Log 2MinTrafoSize, the countCount_mACT is set to +4. Further, in a case in which the size log2TrafoSize of the processing target TU is a size greater by two stepsthan Log 2MinTrafoSize, the count Count_mACT is set to +16.

When the CU is segmented into a plurality of TUs, the foregoingrecursive quad-tree segmentation is executed. Therefore, one TU issegmented into 2×2 TUs. That is, a size of the TU greater by one stepthan the TU with the minimum size is 4 times the TU of the minimum size.Accordingly, in a case in which the size of the processing target TU isgreater by one step than the minimum size, the influence of theprocessing target TU on the entire screen is considered to be 4 timesthe influence of the TU with the minimum size. Accordingly, in a case inwhich the size of the processing target TU is greater by one step thanthe minimum size, the count Count_mACT in the foregoing processingtarget TU is set to +4.

A size of the TU greater by two steps than the TU with the minimum sizeis 4 times the TU greater by one step than the TU with the minimum size.Accordingly, the size of the TU greater by two step than the TU with theminimum size is 16 times the TU with the minimum size. Accordingly, in acase in which the size of the processing target TU is greater by twosteps than the minimum size, the count Count_mACT in regard to theforegoing processing target TU is set to +16.

The position of the processing target TU in the picture can becalculated from the position of a coding tree block (CTB) including theTU, the position of the CU in the CTB, and a depth of the TU. When(xCtb, yCtb) is the position of the CTB, positions (x, y) of 4 TUs atthe time of quad-tree segmentation of the CTB into the CUs are expressedby Equation (4) below. In the Equation (4), the variable “CtbAddrInRs”denotes a coding tree block address in coding tree block raster scan ofa picture.

$\begin{matrix}\left. \begin{matrix}{\left( {x,y} \right) = \left\{ {\left( {{x\; 0},{y\; 0}} \right),\left( {{x\; 1},{y\; 0}} \right),\left( {{x\; 0},{y\; 1}} \right),\left( {{x\; 1},{y\; 1}} \right)} \right\}} \\{{{{xCtb} - \left( {{CtbAddrInRs}\mspace{14mu}\%\mspace{14mu}{PicWidthInCtbsY}} \right)} ⪡ {{Ctb}\;{Log}\; 2\;{SizeY}}};} \\{{{yCtb} = {\left( {{CtbAddrInRs}/{PicWidthInCtbsY}} \right) ⪡ {{Ctb}\;{Log}\; 2\;{SizeY}}}};} \\{{{x\; 0} = {xCtb}};} \\{{{y\; 0} = {yCtb}};} \\{{{x\; 1} - {x\; 0} + \left( {1 ⪡ \left( {{\log\; 2{CtbSize}} - 1} \right)} \right)};} \\{{{y\; 1} = {{y\; 0} + \left( {1 ⪡ \left( {{\log\; 2{CtbSize}} - 1} \right)} \right)}};}\end{matrix} \right\} & (4)\end{matrix}$

The ACT map generation unit 126 executes, for example, the samecalculation process as Equation (5) below as the ACT count process ofstep S2204 in FIG. 8. In the Equation (5), the conditional expression“beginning picture” denotes “xCtb==0 && yCtb==0”.

$\begin{matrix}\left. \begin{matrix}{{{{if}\mspace{14mu}\left( {{beginning}\mspace{14mu}{picture}} \right)\mspace{14mu}{count\_ mACT}} = 0};} \\{{if}\mspace{14mu}\left( {{ACT}==1} \right)\mspace{14mu}\{} \\{\mspace{14mu}{{for}\mspace{14mu}\left( {{i = 0};{i < {{\log\; 2{TrafoSize}} - {{Log}\; 2{Min}\;{TrafoSize}}}};{i++}} \right)\mspace{14mu}\{}} \\{\mspace{31mu}{{for}\mspace{14mu}\left( {{i = 0};{i < {{\log\; 2{TrafoSize}} - {{Log}\; 2{Min}\;{TrafoSize}}}};{i++}} \right)\mspace{14mu}\{}} \\{{{{{ACT\_ map}\left\lbrack {x + i} \right\rbrack}\left\lbrack {y + j} \right\rbrack} = 1};} \\\} \\\} \\{\mspace{14mu}{{{count\_ mACT}+={2^{̑}\left( {{\log\; 2{TrafoSize}} - {{Log}\; 2{Min}\;{TrafoSize}}} \right)}};}} \\{\}\mspace{14mu}{else}\mspace{14mu}\{} \\{\mspace{14mu}{{for}\mspace{14mu}\left( {{i = 0};{i < {{\log\; 2{TrafoSize}} - {{Log}\; 2{Min}\;{TrafoSize}} + 1}};++} \right)\mspace{14mu}\{}} \\{\mspace{31mu}{{for}\mspace{14mu}\left( {{i = 0};{i < {{\log\; 2{TrafoSize}} - {{Log}\; 2{Min}\;{TrafoSize}} + 1}};++} \right)\mspace{14mu}\{}} \\{{{{{ACT\_ map}\lbrack i\rbrack}\lbrack j\rbrack} = 0};} \\\} \\\} \\{\mspace{14mu}{{{count\_ mACT}+=0};}} \\\}\end{matrix} \right\} & (5)\end{matrix}$

FIG. 9 is a flowchart illustrating a method of determining whether themACT is turned on or off based on an ACT map.

The mACT determination unit 125 in the moving picture encoding apparatus1 according to the present embodiment determines whether the mACT isturned on or off based on the ACT map, as described above. Asillustrated in FIG. 9, the mACT determination unit 125 first reads theON/OFF information of the ACT in regard to the reference image used inthe prediction from the ACT map (step S1201). In step S1201, forexample, the mACT determination unit 125 reads the ON/OFF information ofthe ACT in regard to the reference block at the same position as theprediction block in the reference picture.

Next, the mACT determination unit 125 calculates an ACT ON ratio in thereference image as an mACT evaluation value (step S1202). The ACT ONratio is a value indicating a ratio of a region encoded by turning onthe ACT in the reference image (reference block).

Next, the mACT determination unit 125 determines whether the mACTevaluation value is equal to or greater than a threshold value (stepS1203). In a case in which the mACT evaluation value is equal to orgreater than the threshold value (Yes in step S1203), the mACTdetermination unit 125 sets the mACT to be turned on for the prediction(step S1204). In a case in which the mACT evaluation value is less thanthe threshold value (No in step S1203), the mACT determination unit 125sets the mACT to be turned off for the prediction (step S1205).

The ACT ON ratio calculated as the mACT evaluation value is a valueindicating a ratio of a region encoded by turning on the ACT in thereference image (reference block), as described above. Therefore, as thesize of the TU encoded by turning on the ACT in the reference image islarger, the mACT evaluation value is larger. In the case in which theACT is turned on, meaningful information can be concentrated on onecomponent when the encoding is executed with the YCgCo format more thanwhen the encoding is executed with the RGB format. In other words, asthe mACT evaluation value indicates a larger value, the degree ofdeviation in information regarding the R, G, and B components in a videois lower. Conversely, as the mACT evaluation value indicates a lowervalue, the degree of deviation in the information regarding the R, G,and B components in the video is higher. Accordingly, in a case in whichthe mACT evaluation value is large (in the case in which the ACT ONratio is high), it is considered that the meaningful information can beconcentrated on one component for the prediction when the prediction isexecuted by transforming the RGB format into the YCgCo format more thanwhen the prediction is executed while maintaining the RGB format.Accordingly, in the case in which the mACT evaluation value is equal toor greater than the threshold value (Yes in step S1203), the mACTdetermination unit 125 sets the mACT to be turned on and causes theintra prediction unit 101 and the inter prediction unit 102 to executethe prediction using the image data with the YCgCo format.

Hereinafter, examples of a method of calculating the ACT ON ratio in thecase in which the ACT ON ratio is used as the mACT evaluation value willbe described.

FIG. 10 is a diagram illustrating a first example of the method ofcalculating the ACT ON ratio.

In the first example of the method of calculating the ACT ON ratio,ON/OFF information of the ACT in regard to a reference block 201 at thesame position as a prediction block (PU) 301 of a processing targetpicture 3 in a reference picture 2 is used, as illustrated in FIG. 10.Referring to the information regarding the reference block at the sameposition as the prediction block in the inter prediction meanssimplicity of calculation at the time of searching for a similarposition to the prediction block, and thus the referring is considerablyvalued.

In the inter prediction, a motion vector referring to an encoded pictureis searched for. The motion vector is searched for based on the PU unitset in the CU separately from the TU. Therefore, the size of the PU isdifferent from the size of the TU (where PU size TU size) and the motionvector is not suitable for a grid of the processing block (CU) in manycases. Accordingly, as illustrated in FIG. 10, the reference block 201having the same size and located at the same position as the predictionblock 301 of the processing block (CU) 300 include a plurality of TUs201 a, 201 b, 201 c, and 201 d in many cases. In the cases, the mACTdetermination unit 125 adds all of the pieces of information regardingthe ACT map overlapping the reference block 201 and determines whetherthe mACT is turned on or off.

When the mACT is determined to be turned on or off, as described above,the mACT evaluation value (the ACT ON ratio) is used. Therefore, themACT determination unit 125 calculates an ACT ON ratio ACTeval in thereference block 201 of the reference picture 2 as an area ratio using,for example, Equation (6) below.

$\begin{matrix}{{ACTeval} = \frac{\sum\limits_{a,b}^{\;}\left( {{{ACT\_ map}\left\lbrack {a,b} \right\rbrack} \times {{Num}\left\lbrack {a,b} \right\rbrack}} \right)}{\sum\limits_{a,b}^{\;}{{Num}\left\lbrack {a,b} \right\rbrack}}} & (6)\end{matrix}$

In Equation (6), a and b indicate positions in the horizontal andvertical directions of the subblock (TU) included in the reference blockin the CU of which coordinates of the upper left corner are (x, y).Num[a, b] indicates an area of the subblock at the position (a, b)included in the reference block or the number of pixels included in thesubblock.

That is, the mACT determination unit 125 compares the ACT ON ratioACTeval (0≤ACTeval≤1) calculated using Equation (6) to a determinationthreshold value TH1. In a case of ACTeval≥TH1, the mACT is set to beturned on.

FIG. 11 is a diagram illustrating a second example of the method ofcalculating the ACT ON ratio.

In the second example of the method of calculating the ACT ON ratio, asillustrated in FIG. 11, the ON/OFF information of the ACT within asearch range 202 of a motion vector set in the reference picture 2 inthe inter prediction is used. A reference destination of the motionvector in the inter prediction exists within the search range 202 of themotion vector set in the reference picture. The reference destination ofthe motion vector is a block that has high correlation with a currentprocessing block 300. Therefore, in a case in which the search range 202of the motion vector is all encoded in one state of the ON and OFFstates of the ACT, prediction precision is considered to be good whenthe setting of the mACT of the current processing block 300 isconfigured to be the same as the setting of the ACT at the time ofencoding of the search range.

In the first example, the reference block 200 of the reference picture 2having the same size and located at the same position as the currentprocessing block 300 of the current processing picture 3 is set as anevaluation target to calculate an mACT evaluation value. In the secondexample, however, a search range 202 of a motion vector with respect tothe current processing block 300 when the inter prediction unit 102executes motion searching is set as an evaluation target. Therefore, inthe second example, an area to be evaluated is larger than in the firstexample. That is, while the subblock (a, b) in the CU is evaluated inthe first example, the CU is also counted as a search range in thesecond example.

Here, in a case in which horizontal −H to +H−1 and vertical −V to +V−1are set centering on an ACT map (for example, (0, 0)) of a search rangeof a current processing block, a current block size is set to 0 h×0 v,[−H, −V] is set as upper left, and a result of the ACT map in an area of(2 H+0 h)×(2V+0 v) is used for determination. That is, basically, thereference block 200 at the same position as the current processing block300 in the first example is changed to the search range 202 of thecurrent processing block 300 in the second example. Therefore, in thesecond example, the mACT determination unit 125 calculates the ACT ONratio ACTeval in a reference destination block as an area ratio using,for example, Equation (7) below.

$\begin{matrix}{{ACTeval} = \frac{\sum\limits_{x,y}^{\;}{\sum\limits_{a,b}^{\;}\left( {{{ACT\_ map}\left\lbrack {{x + a},{y + b}} \right\rbrack} \times {{Num}\left\lbrack {{x + a},{y + b}} \right\rbrack}} \right)}}{\sum\limits_{x,y}^{\;}{\sum\limits_{a,b}^{\;}{{Num}\left\lbrack {{x + a},{y + b}} \right\rbrack}}}} & (7)\end{matrix}$

In Equation (7), x and y are positions in the horizontal and verticaldirections in the search range 202.

As in the first embodiment, referring to the information regarding thereference block at the same position as the prediction block in theinter prediction means simplicity of calculation at the time ofsearching for a similar position to the prediction block, and thus thereferring is considerably valued. Therefore, in the second example, forexample, the ACT ON ratio may be calculated by weighting an ACTdetermination result in the reference block 200 at the same position asthe current processing block 300 of the current processing picture 3included in the search range 202 of the motion vector of the referencepicture 2. In the second example, instead of the reference block 200 atthe same position as the current processing block 300 of the currentprocessing picture 3 included in the search range 202, for example, areference block at a position deviated from the same position by themagnitude of a separately obtained global vector may be weighted.

FIG. 12 is a diagram illustrating a third example of the method ofcalculating the ACT ON ratio.

In the third example of the method of calculating the ACT ON ratio, amotion vector at the time of encoding of a reference block at the sameposition as a current prediction block is used to calculate a motionvector indicating a current picture through ratio calculation from thereference block at the same position. Then, based on the motion vectorindicating the calculated current picture, a reference block to be usedto calculate an ACT ON ratio is decided.

When the inter prediction unit 102 executes inter frame prediction, forexample, as illustrated in FIG. 12, prediction is executed in some casesusing a second reference picture 4 temporally later than a firstreference picture 2 and temporally earlier than a current processingpicture 3. Here, a reference block 211 in the first reference picture 2and a reference block 412 in the second reference picture 4 are assumedto have high correlation (similarity). In this case, a motion vector 501from the reference block 211 of the first reference picture 2 to thereference block 412 of the second reference picture 4 can be expressedas a motion vector MV_ref(x, y) owned by a block 411 at the sameposition as the reference block 211 of the first reference picture 2 inthe second reference picture 4. When the motion vector MV_ref(x, y) isgiven, a motion vector MV_base(x, y) owned by a block 311 of the currentprocessing picture 3 at the same position as the reference block 211 ofthe first reference picture 2 can be calculated by Equation (8) below.MV_base(x,y)=MV_ref(x,y)×(tb/td)  (8)

In Equation (8), tb and td are a temporal distance from the firstreference picture 2 to the current processing picture 3 and a temporaldistance from the first reference picture 2 to the second referencepicture 4.

The motion vector MV_base(x, y) calculated by Equation (8) is oppositein direction in a case in which the current processing picture 3 isindicated from the reference picture and in a case in which thereference picture is indicated from the current processing picture 3.The motion vector MV_base(x, y) is a motion vector owned by theprocessing block 311 of the current processing picture 3 at the sameposition as the reference block 211 of the first reference picture 2. Onthe other hand, a block which has high correlation (similarity) with thereference block 211 of the first reference picture 2 in another pictureis a reference block present in a direction expressed by the motionvector 501 illustrated in FIG. 12. That is, a block which has highcorrelation (similarity) with the reference block 211 of the firstreference picture 2 in the current processing picture 3 is theprocessing block 301 centering on a position (xp, yp). Therefore, in acase in which the mACT is determined to be turned on or off in the thirdexample, the mACT determination unit 125 may refer to the ACT map from aposition deviated by the magnitude of −MV_base(x, y) from the referenceblock 211 in the first reference picture 2. That is, the method ofcalculating the ACT ON ratio in the third example is basically the sameas that of the first example, but a position at which the ACT map isreferred to is a position deviated by −MV_base(x, y).

In this way, in the third example, the ACT ON ratio is calculatedreferring to the ACT map in consideration of a motion between thecurrent processing picture and the reference picture. Therefore, it ispossible to determine whether the mACT is turned on or off moreappropriately than in the first example.

In the third example, instead of calculating the deviation in the motionbased on the motion vector owned by the blocks at the same positionbetween the pictures, a global vector obtained from motion vectors ofthe entire reference picture may be used. In the third embodiment, forexample, either a motion vector or a global vector owned by blocks atthe same position between pictures may be selected according to thecharacteristics of processing target moving picture data (picture), anda reference block to be used to calculate an ACT ON ratio may beobtained.

FIG. 13 is a diagram illustrating a fourth example of the method ofcalculating the ACT ON ratio.

In the fourth example of the method of calculating the ACT ON ratio, allof the blocks of a reference picture are set as targets and a referenceblock to be used to calculate an ACT ON ratio is selected from referenceblocks in which a motion vector scaled by time division refers to acurrent processing block.

For example, as illustrated in FIG. 13, a first reference block 213 anda second reference block 214 exist as reference blocks referring to aprocessing block 311 of a current processing picture 3 in a firstreference picture 2 in some cases. In the cases, a reference block to beused to calculate the ACT ON ratio is selected based on a motion vectorMV_base in the current processing picture 3 calculated using thereference blocks 213 and 214.

First, in a case in which the first reference block 213 in the firstreference picture 2 is selected, a motion vector owned by a referenceblock 413 in a second reference picture 4 is assumed to be MV_refA. Atthis time, a motion vector owned by the processing block 311 of thecurrent processing picture 3 can be calculated in the same point of viewas the third example. That is, the motion vector can be calculated usinga temporal distance td from the first reference picture 2 to the secondreference picture 4, a temporal distance tb from the current processingpicture 3 to the second reference picture 4, and motion vectors in thereference pictures.

Further, in a case in which the second reference block 214 in the firstreference picture 2 is selected, a motion vector owned by a referenceblock 414 in a second reference picture 4 is assumed to be MV_refB. Atthis time, a motion vector owned by the processing block 311 of thecurrent processing picture 3 can be calculated in the same point of viewas the third example.

In this way, in the fourth example, in a case in which there are theplurality of reference blocks referring to the current processing block,the reference block to be used to calculate the ACT ON ratio is selectedbased on the motion vector MV_base in the current processing picturecalculated using each reference block. Accordingly, in the fourthexample, the block having higher correlation with the current processingblock can be selected as the reference block to be used to calculate theACT ON ratio. In particular, when the reference block having a motionvector closest to a global vector is selected using the global vector,the better reference block can be considered to be selected. After thereference block is selected, the ACT ON ratio can be calculated as inthe third example.

In the fourth example, the block referring to the current processingblock is not present in the reference picture. Therefore, in the fourthexample, any combination of the first to third examples may be used.

As described above, in the encoding for the moving picture according tothe present embodiment, the mACT is determined to be turned on or offfor each processing block or prediction block in a case in which thepicture with the RGB format is input and one component is mainly usedamong three components of the color space for the prediction. In thecase in which the mACT is turned on, the intra prediction unit 101 andthe inter prediction unit 102 executes the prediction using the imagedata of which the color space is transformed from the RGB format to theYCgCo format. In the case in which the mACT is turned off, the intraprediction unit 101 and the inter prediction unit 102 executes theprediction using the image data of which the color space is the RGBformat.

Therefore, in the encoding for the moving picture according to thepresent embodiment, the format of the higher precision can be selectedbetween the RGB format and the YCgCo format according to thecharacteristics of the color space information in the input picture toexecute the prediction. For example, in a case in which the meaningfulinformation is averagely included in each of the components R, G, and B,the meaningful information of each of the components R, G, and B can beconcentrated on the luminance component Y by transforming the RGB formatinto the YCgCo format. In a case in which the meaningful information isconcentrated on any one of the three R, G, and B components, it ispossible to suppress the meaningful information of one component frombeing reduced by executing the prediction while maintaining the RGBformat.

In the present embodiment, as described above, the mACT is determined tobe turned on or off for each processing block or prediction block usingthe ACT map. Therefore, in a case in which a picture includes aplurality of regions in which balance of the RGB components in thepicture is different, the mACT can be determined to be turned on or offaccording to the position of the processing block or the predictionblock. For a prediction block in which there are many TUs encoded withthe YCgCo format in the reference block, the prediction can be executedwith the YCgCo format by turning on the mACT in the YCgCo format. Thatis, by determining whether the mACT is turned on or off using the ACTmap, it is possible to execute the prediction suitable for the colorspace format when the reference picture is encoded. Thus, it is possibleto execute the prediction with higher precision in the encoding for themoving picture according to the present embodiment.

Thus, according to the present embodiment, it is possible to execute theprediction with higher precision while suppressing an increase in acalculation amount. Accordingly, according to the present embodiment, itis possible to improve a capability to encode the moving picture.

The flowcharts of FIGS. 7A, 7B, 8, and 9 are merely examples. Content ofsome of the processes may be modified as inevitable or other processesmay be added.

The ACT ON ratio used as the mACT evaluation value is not limited to theforegoing first to fourth examples, but may be calculated based on theON/OFF setting of the ACT in a specific reference block in accordancewith another method. The ACT ON ratio is merely an example of the mACTevaluation value. The mACT evaluation value can be calculated using theACT map may be another value by which it is possible to determine eitherthe RGB format or the YCgCo format to execute the prediction with highprecision.

FIG. 14 is a diagram illustrating a functional configuration of a movingpicture encoding apparatus according to a third embodiment.

As illustrated in FIG. 14, a moving picture encoding apparatus 1according to the present embodiment includes an intra prediction unit101, an inter prediction unit 102, a determination unit 103, a predictedimage generation unit 104, and a prediction error signal generation unit105. The moving picture encoding apparatus 1 includes a first colorspace selection unit 106, a T/Q processing unit 107, and an ENTprocessing unit 108. The moving picture encoding apparatus 1 furtherincludes an IQ/IT processing unit 109, a color space restoration unit110, a decoded image generation unit 111, a filter processing unit 112,and a frame memory 113. The moving picture encoding apparatus 1according to the present embodiment further includes an mACTdetermination unit 125, a second color space selection unit 122, areference image acquisition unit 131, a YUV transform unit 132, and anACT map generation unit 126.

The intra prediction unit 101, the inter prediction unit 102, thedetermination unit 103, the predicted image generation unit 104, and theprediction error signal generation unit 105 in the moving pictureencoding apparatus 1 according to the present embodiment have theforegoing respective functions. The first color space selection unit 106in the moving picture encoding apparatus 1 has the same function as theforegoing first color space selection unit 106 (see FIG. 1). The T/Qprocessing unit 107, the ENT processing unit 108, the IQ/IT processingunit 109, the color space restoration unit 110, the decoded imagegeneration unit 111, the filter processing unit 112, and the framememory 113 in the moving picture encoding apparatus 1 have the foregoingrespective functions. The frame memory 113 in the moving pictureencoding apparatus 1 according to the present embodiment stores an RGBdecoded image 113 a, a YUV decoded image 113 c, and an ACT map 113 b.The RGB decoded image 113 a and the YUV decoded image 113 c are adecoded image with the RGB format and a decoded image with the YUVformat, respectively.

The mACT determination unit 125 of the moving picture encoding apparatus1 according to the present embodiment determines which image data witheither the RGB format or the YUV format is used for prediction, in otherwords, determines whether the ACT (mACT) for prediction is turned on oroff. The mACT determination unit 125 according to the present embodimentdetermines whether the mACT is turned on or off based on an ACT map 113d stored in the frame memory 113, as described in the second embodiment.The mACT determination unit 125 outputs an ON/OFF determination resultof the mACT to the second color space selection unit 122 and thereference image acquisition unit 131.

In a case in which a picture (moving picture data) input to the movingpicture encoding apparatus 1 has the RGB format and one component ismainly used among three components for the prediction, the mACTdetermination unit 125 determines which image data with either the RGBformat or the YUV (YCgCo) format is used for prediction. That is, in acase in which the moving picture data input to the moving pictureencoding apparatus 1 has the YUV format, the mACT determination unit 125determines that the prediction is executed using the image data with theYUV format (determines that the mACT is turned on). The mACTdetermination unit 125 determines whether the input moving picture datahas the RGB format or the YUV format by, for example, a control signalfrom an overall control unit (not illustrated in FIG. 14) that controlsan overall operation of the moving picture encoding apparatus 1.

The second color space selection unit 122 selects a color space oforiginal image data input to the intra prediction unit 101 and the interprediction unit 102 based on a determination result of the mACTdetermination unit 125. The second color space selection unit 122includes an ACT unit 122 a and a switch (not illustrated). The ACT unit122 a transforms the color space of the original image data from the RGBformat to the YCgCo format. Whether the original image data with the RGBformat input to the second color space selection unit 122 is outputwhile maintaining the RGB format or the original image data with the RGBformat is transformed into the YCgCo format by the ACT unit 122 a isswitched by the switch.

The reference image acquisition unit 131 selects the color space formatof the reference image data to be input to the intra prediction unit 101and the inter prediction unit 102 based on the determination result ofthe mACT determination unit 125 and acquires the reference image datawith the selected color space format. In the case in which the mACT isturned off, the reference image acquisition unit 131 acquires thereference image data from the RGB decoded image 113 a of the framememory 113. Conversely, in the case in which the mACT is turned on, thereference image acquisition unit 131 acquires the reference image datafrom the YUV decoded image 113 c of the frame memory 113.

The YUV transform unit 132 transforms the color space format of thedecoded image subjected to the filter process by the filter processingunit 112 from the RGB format to the YCgCo format. The YUV transform unit132 stores the decoded image (that is, the YUV decoded image 113 c)transformed into the YCgCo format in the frame memory 113.

The ACT map generation unit 126 generates an ACT map including screenposition information and the number of counts by counting whetherencoding (orthogonal transform and quantization) is executed using a oneprocess between ACT ON and ACT OFF for each picture. The ACT mapgeneration unit 126 of the moving picture encoding apparatus 1 accordingto the present embodiment generates an ACT map, for example, byexecuting the ACT map generation process (see FIG. 8) described in thesecond embodiment. The ACT map generation unit 126 stores the generatedACT map 113 b in the frame memory 113.

FIG. 15A is a flowchart (part 1) illustrating an encoding processaccording to the third embodiment. FIG. 15B is a flowchart (part 2)illustrating the encoding process according to the third embodiment.FIGS. 15A and 15B illustrate flowcharts of an encoding process on oneprocessing block in a case in which the picture (the original imagedata) with the RGB format is input and the prediction is executed mainlyusing one component among three components of the color space. Theprediction executed using mainly one component among three components ofthe color space is prediction of a case in which a flag“separate_colour_plane_flag” present in the SPS header is“separate_colour_plane_flag=0”, as described in the first embodiment.For example, the overall control unit (not illustrated in FIG. 14)controlling an overall operation of the moving picture encodingapparatus 1 determines whether the input picture has the RGB format. Theoverall control unit determines whether the prediction is executedmainly using one component among three components, for example, by avalue of the flag “separate_colour_plane_flag”.

In a case in which the prediction is executed on the input picture withthe RGB format mainly using one component among three components of thecolor space, the moving picture encoding apparatus 1 first determineswhether there is the ACT map usable for the ON/FF determination of themACT, as illustrated in FIG. 15A (step S11). The mACT determination unit125 executes step S11. In a case in which there is the ACT map usablefor the determination (Yes in step S11), the mACT determination unit 125determines whether the mACT is turned on or off based on the ACT map(step S12). Conversely, in a case in which there is no ACT map usablefor the determination (No in step S11), the mACT determination unit 125determines whether the mACT is turned on or off based on initial setting(step S13). The mACT determination unit 125 outputs the determinationresult of step S12 or S13 to the second color space selection unit 122and the reference image acquisition unit 131. In step S12, the mACT isdetermined to be turned on or off based on the ACT ON ratio (the mACTevaluation value) calculated in accordance with, for example, the samemethod described in the second embodiment. The initial setting in thedetermination of step S13 is, for example, setting in which the mACT isnormally turned on or setting in which the determination is executedusing variances or the like of the RGB components, as in the firstembodiment.

Thereafter, in the moving picture encoding apparatus 1, a predictionprocess is executed based on the ON/OFF determination result of the mACT(step S14). The prediction process of step S14 is executed incooperation with the second color space selection unit 122, thereference image acquisition unit 131, the intra prediction unit 101, theinter prediction unit 102, and the determination unit 103.

In a case in which the mACT is turned on, the second color spaceselection unit 122 outputs the original image data transformed into theYCgCo format by the ACT unit 122 a to the intra prediction unit 101 andthe inter prediction unit 102. In the case in which the mACT is turnedon, the reference image acquisition unit 131 outputs the reference imagedata read from the decoded image (the YUV decoded image 113 c) with theYCgCo format in the frame memory 113 to the intra prediction unit 101and the inter prediction unit 102.

Conversely, in a case in which the mACT is turned off, the second colorspace selection unit 122 outputs the input original image data with theRGB format to the intra prediction unit 101 and the inter predictionunit 102, while maintaining the RGB format. In the case in which themACT is turned on, the reference image acquisition unit 131 outputs thereference image data read from the decoded image (the RGB decoded image113 a) with the RGB format in the frame memory 113 to the intraprediction unit 101 and the inter prediction unit 102.

Each of the intra prediction unit 101 and the inter prediction unit 102executes the prediction using the original image data input from thesecond color space selection unit 122 and the reference image data inputfrom the reference image acquisition unit 131. Each of the intraprediction unit 101 and the inter prediction unit 102 executes theprediction based on the prediction block (PU) unit set in the processingblock (CU). When the intra prediction unit 101 and the inter predictionunit 102 end the prediction on each prediction block included in oneprocessing block, the intra prediction unit 101 and the inter predictionunit 102 output the prediction result to the determination unit 103. Thedetermination unit 103 determines a prediction result (predicted image)in which an encoding cost is the minimum based on the prediction resultsof the intra prediction unit 101 and the inter prediction unit 102. Thedetermination unit 103 outputs the determination result, that is, theprediction result in which the encoding cost is the minimum, to thepredicted image generation unit 104. Accordingly, the prediction processof step S14 on one processing block (CU) ends.

When the prediction process of step S14 ends, the predicted imagegeneration unit 104 subsequently generates a predicted image with theRGB format based on the prediction result (step S15). The predictedimage generation unit 104 outputs the generated predicted image to theprediction error signal generation unit 105. The predicted image is alsoused when a local decoded image is generated. Therefore, the predictedimage generated by the predicted image generation unit 104 is output tothe prediction error signal generation unit 105 and is also stored in,for example, a buffer (not illustrated) included in the decoded imagegeneration unit 111 or the like.

After the predicted image is generated, the prediction error signalgeneration unit 105 in the moving picture encoding apparatus 1 generatesa prediction error signal (step S16). The prediction error signalgeneration unit 105 generates the prediction error signal with the RGBformat using the original image data and the predicted image with theRGB format. The prediction error signal generation unit 105 outputs thegenerated prediction error signal with the RGB format to the first colorspace selection unit 106.

After the prediction error signal is generated, as illustrated in FIG.15B, the first color space selection unit 106 in the moving pictureencoding apparatus 1 executes the ACT process (step S17). The firstcolor space selection unit 106 determines whether the prediction errorsignal is output to the T/Q processing unit 107 while maintaining theRGB format or the prediction error signal is transformed into the YCgCoformat and is output to the T/Q processing unit 107 for each transformblock (TU) set in the processing block (CU). In a case in which theprediction error signal with the RGB format is output to the T/Qprocessing unit 107, the first color space selection unit 106 outputsthe prediction error signal from the prediction error signal generationunit 105 to the T/Q processing unit 107 without transform. Conversely,in a case in which the prediction error image transformed into the YCgCoformat is output to the T/Q processing unit 107, the first color spaceselection unit 106 transforms the prediction error signal from the RGBformat to the YCgCo format through the inverse transform of Equation (1)in the ACT unit 106 a. When the ACT process on one processing block (CU)ends, the first color space selection unit 106 stores ON/OFF informationof the ACT regarding each transform block to a buffer (not illustrated)included in the color space restoration unit 110 or the like.

When the prediction error signal is input to the T/Q processing unit 107through the ACT process executed by the first color space selection unit106, the T/Q processing unit 107 executes the orthogonal transform andthe quantization based on the transform block (TU) unit (step S18). TheT/Q processing unit 107 determines whether the transform block haseither the RGB format or the YCgCo format and executes the orthogonaltransform and the quantization corresponding to each format. A transformcoefficient quantized by the T/Q processing unit 107 is output to theENT processing unit 108. When the transform coefficient quantized by theT/Q processing unit 107 is input, the ENT processing unit 108 executesan ENT process of executing arithmetic encoding or entropy encoding onthe input transform coefficient to generate a bit stream (step S19).

The transform coefficient quantized by the T/Q processing unit 107 isalso used to generate a reference image (local decoded image) at thetime of encoding of a subsequent processing block or picture. That is,after the orthogonal transform and the quantization of step S18 areexecuted, the moving picture encoding apparatus 1 executes the ENTprocess and also executes a decoded image generation process (step S20)and a filter process (step S21). The decoded image generation process ofstep S20 is executed by the IQ/IT processing unit 109, the color spacerestoration unit 110, and the decoded image generation unit 111. Thefilter process of step S21 is executed by the filter processing unit112. In the decoded image generation process of step S20, the IQ/ITprocessing unit 109 first executes the inverse orthogonal transform andthe inverse quantization on the transform coefficient quantized by theT/Q processing unit 107 to restore the prediction error image before theexecution of the orthogonal transform. Next, the color space restorationunit 110 executes inverse transform on the transform block from theYCgCo format to the RGB format based on ON/OFF information of the ACT inregard to each transform block (TU) in the prediction error image(processing block) to restore the prediction error signal with the RGBformat. Thereafter, the decoded image generation unit 111 generates adecoded image with the RGB format in regard to the original image datausing the prediction error signal restored in the RGB format and thepredicted image generated by the predicted image generation unit 104. Inthe filter process of step S21, for example, the filter processing unit112 executes a filter process on the decoded image generated by thedecoded image generation unit 111. In a case in which a moving pictureis encoded in conformity with the H.265/HEVC standard, for example, thefilter processing unit 112 continuously executes the SAO process afterthe deblocking filter process. When the predetermined filter process onthe decoded image ends, the filter processing unit 112 stores thedecoded image (the RGB decoded image 113 a) with the RGB formatsubjected to the filter process in the frame memory 113.

Further, the filter processing unit 112 in the moving picture encodingapparatus 1 according to the present embodiment inputs the decoded imagewith the RGB format subjected to the filter process to the YUV transformunit 132. The YUV transform unit 132 generates the decoded image withthe YUV format from the input decoded image with the RGB format throughthe inverse transform of Equation (1) (step S23). The YUV transform unit132 stores the generated decoded image (the YUV decoded image 113 c)with the YUV format in the frame memory 113.

In the moving picture encoding apparatus 1 according to the presentembodiment generates the decoded image in step S20, and subsequentlyexecutes the ACT map generation process (step S22) along with the filterprocess (step S21) and the process of generating the decoded image withthe YUV format (step S23). The ACT map generation process of step S22 isexecuted by the ACT map generation unit 126. The ACT map generation unit126 generates the ACT map including screen position information and thenumber of counts in regard to the TU subjected to the orthogonaltransform and quantization by turning on the ACT, based on the ON/OFFinformation of the ACT for each transform block (TU) used by the colorspace restoration unit 110. The ACT map generation unit 126 generatesthe ACT map in accordance with, for example, the method described in thesecond embodiment.

The ACT map generation unit 126 stores the generated ACT map 113 b inthe frame memory 113. The ACT map 113 b stored in the frame memory 113is referred to by the mACT determination unit 125 to determine whetherthe mACT is turned on or off at the time of prediction of the subsequentprocessing block.

As described above, the encoding process on one processing block (CU)ends in the moving picture encoding apparatus 1.

In the moving picture encoding apparatus 1, the processes of steps S11to S23 are sequentially executed on each of the plurality of processingblocks set in one picture. At this time, the moving picture encodingapparatus 1 executes the processes of steps S11 to S23 on eachprocessing block in a pipeline way.

As described above, in the encoding for the moving picture according tothe present embodiment, the mACT is determined to be turned on or offfor each processing block or prediction block in a case in which thepicture with the RGB format is input and one component is mainly usedamong three components of the color space for the prediction. In thecase in which the mACT is turned on, the intra prediction unit 101 andthe inter prediction unit 102 execute the prediction using the imagedata of which the color space is transformed from the RGB format to theYCgCo format. In the case in which the mACT is turned off, the intraprediction unit 101 and the inter prediction unit 102 execute theprediction using the image data of which the color space is the RGBformat.

Therefore, in the encoding for the moving picture according to thepresent embodiment, the format of the higher precision can be selectedbetween the RGB format and the YCgCo format according to thecharacteristics of the color space information in the input picture toexecute the prediction. For example, in a case in which the meaningfulinformation is averagely included in each of the components R, G, and B,the meaningful information of each of the components R, G, and B can beconcentrated on the luminance component Y by transforming the RGB formatinto the YCgCo format. In a case in which the meaningful information isconcentrated on any one of the three R, G, and B components, it ispossible to suppress the meaningful information of one component frombeing reduced by executing the prediction while maintaining the RGBformat.

In the present embodiment, as described above, the mACT is determined tobe turned on or off for each processing block or prediction block usingthe ACT map. Therefore, in a case in which a picture includes aplurality of regions in which balance of the RGB components in thepicture is different, the mACT can be determined to be turned on or offaccording to the position or the size of the processing block (or theprediction block). For a prediction block in which there are many TUsencoded with the YCgCo format in the reference block, the prediction canbe executed with the YCgCo format by turning on the mACT in the YCgCoformat. That is, by determining whether the mACT is turned on or offusing the ACT map, it is possible to execute the prediction suitable forthe color space format when the reference picture is encoded. Thus, itis possible to execute the prediction with higher precision in theencoding for the moving picture according to the present embodiment.

Thus, according to the present embodiment, it is possible to execute theprediction with higher precision while suppressing an increase in acalculation amount. Accordingly, according to the present embodiment, itis possible to improve a capability to encode the moving picture.

The flowcharts of FIGS. 15A and 15B are merely examples. Content of someof the processes may be modified as inevitable or other processes may beadded. For example, the decoded image with the YUV format may begenerated in the decoded image generation process (step S20).

The moving picture encoding apparatus 1 according to the first to thethird embodiments can each be realized by a computer and a program thatcauses the computer to execute the encoding process including theforegoing mACT determination process. Hereinafter, the moving pictureencoding apparatus 1 realized by the computer and the program will bedescribed with reference to FIG. 16.

FIG. 16 is a diagram illustrating a hardware configuration of acomputer. As illustrated in FIG. 16, a computer 9 operated as the movingpicture encoding apparatus 1 includes a central processing unit (CPU)901, a main storage device 902, an auxiliary storage device 903, aninput device 904, and a display device 905. The computer 9 furtherincludes a digital signal processor (DSP) 906, an interface device 907,a storage medium driving device 908, and a communication device 909.These elements 901 to 909 in the computer 9 are connected to each otherby a bus 910 so that data can be transmitted and received between theseelements.

The CPU 901 is an arithmetic processing device that controls anoperation of the entire computer 9 by executing various programsincluded in an operating system.

The main storage device 902 includes a read-only memory (ROM) and arandom access memory (RAM) (neither of which is illustrated). Forexample, a predetermined basic control program read by the CPU 901 atthe time of activation of the computer 9 is recorded in advance in theROM. The RAM is used as a working storage area, as inevitable, when theCPU 901 executes various programs. For example, the RAM of the mainstorage device 902 can be used to temporarily store the currentprocessing target picture (the original image data), the local decodedimage, the ON/OFF information of the ACT, the ACT map, and the like.

The auxiliary storage device 903 is a storage device, such as a harddisk drive (HDD) or a solid state drive (SSD), that has a largercapacity than the main storage device 902. Various kinds of data orvarious programs to be executed by the CPU 901 can be stored in theauxiliary storage device 903. Examples of the programs stored in theauxiliary storage device 903 include application programs which executeencoding or reproduction of moving picture data and programs whichgenerate (create) moving picture data. Examples of the data stored inthe auxiliary storage device 903 include encoding target moving picturedata and encoded moving picture data.

The input device 904 is, for example, a keyboard device or a mousedevice. When the input device 904 is operated by an operator of thecomputer 9, input information associated with operation content istransmitted to the CPU 901.

The display device 905 is, for example, a liquid crystal display. Thedisplay device 905 displays various kinds of text, images, and the likeaccording to display data transmitted from the CPU 901 or the like.

The DSP 906 is an arithmetic processing device that executes some of theprocesses in the moving picture data encoding process according tocontrol signals or the like from the CPU 901.

The interface device 907 is an input and output device that connects thecomputer 9 to another electronic apparatus to enable transmission andreception of data between the computer 9 and the other electronicapparatus. The interface device 907 includes, for example, a terminalwhich can connect a cable having a connector of a Universal Serial Bus(USB) standard and a terminal which can connect a cable having aconnector of a High-Definition Multimedia Interface (HDMI (registeredtrademark)) standard. An example of the electronic apparatus connectedto the computer 9 by the interface device 907 includes an imagingapparatus such as a video camera.

The storage medium driving device 908 reads a program or data recordedin a portable storage medium (not illustrated) and writes data stored inthe auxiliary storage device 903 on a portable storage medium. As theportable storage medium, for example, a flash memory having a connectorof a USB standard can be used. As the portable storage medium, anoptical disc such as a compact disk (CD), a digital versatile disc(DVD), or a Blu-ray disc (where Blu-ray is a registered trademark) canbe used.

The communication device 909 is a device that connects the computer 9 toa communication network such as the Internet or a local area network(LAN) so that communication can be executed and control thecommunication with another communication terminal (computer) via thecommunication network. The computer 9 can transmit and receive encodedmoving picture data (a bit stream) to another communication terminal viathe communication device 909 and the communication network.

In the computer 9, the CPU 901 reads a program including theabove-described encoding process from the auxiliary storage device 903or the like and executes an encoding process and a decoding process onthe moving picture data in cooperation with the DSP 906, the mainstorage device 902, the auxiliary storage device 903, and the like. Atthis time, the CPU 901 causes the DSP 906 to execute arithmeticprocesses such as the ON/OFF determination of the mACT, the predictionprocess based on the determination result, the orthogonal transform andthe quantization subjected to the prediction process, and the entropyencoding, and the decoding process.

For example, the moving picture data (encoded bit stream) encoded by thecomputer 9 can be transmitted to another computer or the like via theforegoing communication network. The moving picture data encoded by thecomputer 9 can also be stored in the auxiliary storage device 903 to bedecoded (reproduced) by the computer 9 as inevitable. Further, themoving picture data encoded by the computer 9 can also be stored in arecording medium a recording medium using the storage medium drivingdevice 908 for distribution.

The computer 9 used as the moving picture encoding apparatus 1 may notnecessarily include all of the constituent elements illustrated in FIG.15, but some of the constituent elements may also be omitted accordingto use purposes or conditions. For example, in a case in which theprocessing capability of the CPU 901 is high, the DSP 906 may be omittedand the foregoing encoding and decoding processes may be executed by theCPU 901. In a case in which the moving picture data encoding process orthe like is configured to be executed by the DSP 906, for example, anexternal memory that stores the moving picture data to be subjected tothe encoding process may be installed apart from the foregoing mainstorage device (RAM) 902 or auxiliary storage device 903.

The computer 9 is not limited to a general-purpose computer thatrealizes a plurality of functions by executing various programs, but maybe a dedicated information processing apparatus specialized for a movingpicture encoding process. Further, the computer 9 may also be adedicated information processing apparatus specialized for a movingpicture encoding process and an encoded moving picture decoding process.

All examples and conditional language recited herein are intended forpedagogical purposes to aid the reader in understanding the inventionand the concepts contributed by the inventor to furthering the art, andare to be construed as being without limitation to such specificallyrecited examples and conditions, nor does the organization of suchexamples in the specification relate to a showing of the superiority andinferiority of the invention. Although the embodiments of the presentinvention have been described in detail, it should be understood thatthe various changes, substitutions, and alterations could be made heretowithout departing from the spirit and scope of the invention. Forexample, the steps recited in any of the process or method descriptionsmay be executed in any order and are not limited to the order presented.

What is claimed is:
 1. An apparatus for encoding a moving picture, theapparatus comprising: a memory; and a processor coupled to the memoryand configured to execute a determination process that includes,determining either color space format between an RGB format and a YUVformat in a case in which a prediction selecting one component amongthree components in a color space format is designated in input movingpicture data with the RGB format, in which the RGB format is selectedwhen statistical information regarding each of R, G, and B components inthe moving picture data with the RGB format is less than a thresholdvalue and the YUV format is selected when the statistical information isgreater than or equal to the threshold value, execute a selectionprocess, the selection process including selecting either color spaceformat between the RGB format and the YUV format, in which a predictionerror signal for the moving picture data with the RGB format is encodedbased on a determination result of the determination process, andexecute a transform process, the transform process including executingorthogonal transform and quantization on the prediction error signal inthe selected color space format and generating an encoded bit streamusing a value subjected to the orthogonal transform and thequantization.
 2. The apparatus according to claim 1, wherein thedetermination process includes determining either color space formatbetween the RGB format and the YUV format, in which the intra predictionand the inter prediction are executed based on a magnitude of a varianceof each of the R, G, and B components in the moving picture data withthe RGB format.
 3. The apparatus according to claim 1, wherein theprocessor is configured to execute a predicted image generation process,the predicted image generation process including generating a predictedimage based on a prediction result of the intra prediction and the interprediction executed for each processing block included in a processingtarget picture in the moving picture data, execute a prediction errorsignal generation process, the prediction error signal generationprocess including generating a prediction error signal based on theprocessing block and the predicted image, execute a first transformprocess, the first transform process including executing the orthogonaltransform and the quantization on the prediction error signal witheither the RGB format or the YUV format for each subblock set in theprocessing block, execute a second transform process, the secondtransform process including restoring the prediction error signal byexecuting inverse quantization and inverse orthogonal transform on thevalue quantized through the first transform process, execute a colorspace restoration process, the color space restoration process includingtransforming the subblock subjected to the orthogonal transform and thequantization in the YUV format by the first transform process into theRGB format, and execute an ACT map generation process, the ACT mapgeneration process including generating an ACT map including informationregarding a position of the subblock and the number of subblockstransformed from the YUV format to the RGB format by executing the colorspace restoration process, and wherein the determination processincludes executing determining either color space format between the RGBformat and the YUV format to execute the intra prediction and the interprediction based on the ACT map.
 4. The apparatus according to claim 3,wherein the determination process includes determining either colorspace format between the RGB format and the YUV format, in which theintra prediction and the inter prediction are executed on a currentprocessing block based on the ACT map in a reference block at the sameposition as the current processing block in a reference picture which isreferred to in the inter prediction.
 5. The apparatus according to claim3, wherein the determination process includes determining either colorspace format between the RGB format and the YUV format, in which theintra prediction and the inter prediction are executed on a currentprocessing block based on the ACT map within a reference range of thecurrent processing block in a reference picture which is referred to inthe inter prediction.
 6. The apparatus according to claim 3, wherein thedetermination process includes determining either color space formatbetween the RGB format and the YUV format, in which the intra predictionand the inter prediction are executed on a current processing blockbased on an evaluation value which weights the ACT map within apredetermined range including a position deviated by a global vector ora motion vector indicating a block at the same position as the currentprocessing block in a reference block of a reference picture which isreferred to in the inter prediction.
 7. The apparatus according to claim3, wherein the determination process includes setting, as a referenceblock, a processing block in which a motion vector used to encode theprocessing block indicates a direction of a current processing blockamong processing blocks in a reference picture which is referred to inthe inter prediction, and determining either color space format betweenthe RGB format and the YUV format, in which the intra prediction and theinter prediction are executed on the current processing block based onthe ACT map in the reference block.
 8. The apparatus according to claim1, wherein the memory is configured to store a reference image used forthe intra prediction and the inter prediction, and wherein the processoris configured to execute an intra prediction process, the intraprediction process including executing the intra prediction on eachprocessing block, execute an inter prediction process, the interprediction process including executing the inter prediction on eachprocessing block, and execute a color space selection process, the colorspace selection process including transforming input image data into theYUV format and inputting the image data to the intra prediction processand the inter prediction process in a case in which the result ofexecuting the determination process indicates that the YUV format isselected for the intra and inter prediction of the image data, andtransforming reference image data read from the memory into the YUVformat and inputting the reference image data to the intra predictionprocess and the inter prediction process.
 9. The apparatus according toclaim 1, wherein the processor is configured to execute an intraprediction process, the intra prediction process including executing theintra prediction on each processing block, execute an inter predictionprocess, the inter prediction process including executing the interprediction on each processing block, and execute a predicted imagegeneration process, the predicted image generation process includinggenerating a predicted image based on a prediction result of the intraprediction and the inter prediction executed for each processing block,execute a prediction error signal generation process, the predictionerror signal generation process including generating a prediction errorsignal based on the input image data and the predicted image, execute afirst transform process, the first transform process including executingthe orthogonal transform and the quantization on the prediction errorsignal with either the RGB format or the YUV format for each subblockset in the processing block, execute a second transform process, thesecond transform process including restoring the prediction error signalby executing inverse quantization and inverse orthogonal transform onthe value quantized through the first transform process, execute a colorspace restoration process, the color space restoration process includingtransforming the subblock subjected to the orthogonal transform and thequantization in the YUV format in the first transform process among thesubblocks in the restored prediction error signal into the RGB format,execute a decoded image generation process, the decoded image generationprocess including generating a decoded image with the RGB format basedon the prediction error signal in which all of the subblocks arerestored in the RGB format and the predicted image generated through thepredicted image generation process and storing the decoded image in thememory, execute a transform process, the transform process includingtransforming the decoded image with the RGB format into the YUV formatand storing the decoded image in the memory, execute a color spaceselection process, the color space selection process includingtransforming the input image data into the YUV format and inputting theimage data to the intra prediction process and the inter predictionprocess in a case in which it is determined that the intra predictionand the inter prediction are executed on the image data with the YUVformat in the determination process, and execute an acquisition process,the acquisition process including acquiring one of reference images withthe RGB format and the YUV format from the memory according to a resultof executing the determination process and inputting the reference imageto the intra prediction process and the inter prediction process. 10.The apparatus according to claim 1, wherein the YUV format is formed bythree components, a luminance component signal Y, a color differencesignal Cg of a green component, and a color difference signal Co of anorange component.
 11. A method executed by a processor for encoding amoving picture, the method comprising: executing a determination processthat includes, determining either color space format between an RGBformat and a YUV format in a case in which a prediction selecting onecomponent among three components in a color space format is designatedin input moving picture data with the RGB format, in which the RGBformat is selected when statistical information regarding each of R, G,and B components in moving picture data with the RGB format is less thana threshold value, and the YUV format is selected when the statisticalinformation is greater than or equal to the threshold value; executing aselection process, the selection process including selecting eithercolor space format between the RGB format and the YUV format, in which aprediction error signal for the moving picture data with the RGB formatis encoded based on a determination result of the determination process;and executing a transform process, the transform process includingexecuting orthogonal transform and quantization on the prediction errorsignal in the selected color space format and generating an encoded bitstream using a value subjected to the orthogonal transform and thequantization.
 12. The method according to claim 11, wherein thedetermination process includes determining either color space formatbetween the RGB format and the YUV format, in which the intra predictionand the inter prediction are executed based on a magnitude of a varianceof each of the R, G, and B components in the moving picture data withthe RGB format.
 13. The method according to claim 11, the method furthercomprising: executing a predicted image generation process, thepredicted image generation process including generating a predictedimage based on a prediction result of the intra prediction and the interprediction executed for each processing block included in a processingtarget picture in the moving picture data; executing a prediction errorsignal generation process, the prediction error signal generationprocess including generating a prediction error signal based on theprocessing block and the predicted image; executing a first transformprocess, the first transform process including executing the orthogonaltransform and the quantization on the prediction error signal witheither the RGB format or the YUV format for each subblock set in theprocessing block; executing a second transform process, the secondtransform process including restoring the prediction error signal byexecuting inverse quantization and inverse orthogonal transform on thevalue quantized through the first transform process; executing a colorspace restoration process, the color space restoration process includingtransforming the subblock subjected to the orthogonal transform and thequantization in the YUV format by the first transform process into theRGB format; and executing an ACT map generation process, the ACT mapgeneration process including generating an ACT map including informationregarding a position of the subblock and the number of subblockstransformed from the YUV format to the RGB format by the color spacerestoration process, and wherein the determination process includesexecuting determining either color space format between the RGB formatand the YUV format to execute the intra prediction and the interprediction based on the ACT map.
 14. The method according to claim 13,wherein the determination process includes determining either colorspace format between the RGB format and the YUV format, in which theintra prediction and the inter prediction are executed on a currentprocessing block based on the ACT map in a reference block at the sameposition as the current processing block in a reference picture which isreferred to in the inter prediction.
 15. The method according to claim13, wherein the determination process includes determining either colorspace format between the RGB format and the YUV format, in which theintra prediction and the inter prediction are executed on a currentprocessing block based on the ACT map within a reference range of thecurrent processing block in a reference picture which is referred to inthe inter prediction.
 16. The method according to claim 13, wherein thedetermination process includes determining either color space formatbetween the RGB format and the YUV format, in which the intra predictionand the inter prediction are executed on a current processing blockbased on an evaluation value which weights the ACT map within apredetermined range including a position deviated by a global vector ora motion vector indicating a block at the same position as the currentprocessing block in a reference block of a reference picture which isreferred to in the inter prediction.
 17. A non-transitorycomputer-readable storage medium storing a program that causes aprocessor to execute a process, the process comprising; executing adetermination process that includes, determining either color spaceformat between an RGB format and a YUV format in a case in which aprediction selecting one component among three components in a colorspace format is designated in input moving picture data with the RGBformat, in which the RGB format is selected when statistical informationregarding each of R, G, and B components in moving picture data with theRGB format is less than a threshold value, and the YUV format isselected when the statistical information is greater than or equal tothe threshold value; executing a selection process, the selectionprocess including selecting either color space format between the RGBformat and the YUV format, in which a prediction error signal for themoving picture data with the RGB format is encoded based on adetermination result of the determination process; and executing atransform process; the transform process including executing orthogonaltransform and quantization on the prediction error signal in theselected color space format and generating an encoded bit stream using avalue subjected to the orthogonal transform and the quantization. 18.The non-transitory computer-readable storage medium according to claim17, wherein the determination process includes determining either colorspace format between the RGB format and the YUV format, in which theintra prediction and the inter prediction are executed based on amagnitude of a variance of each of the R, G, and B components in themoving picture data with the RGB format.
 19. The non-transitorycomputer-readable storage medium according to claim 17, wherein theprocess comprising: executing a predicted image generation process, thepredicted image generation process including generating a predictedimage based on a prediction result of the intra prediction and the interprediction executed for each processing block included in a processingtarget picture in the moving picture data; executing a prediction errorsignal generation process, the prediction error signal generationprocess including generating a prediction error signal based on theprocessing block and the predicted image: executing a first transformprocess; the first transform process including executing the orthogonaltransform and the quantization on the prediction error signal witheither the RGB format or the YUV format for each subblock set in theprocessing block; executing a second transform process, the secondtransform process including restoring the prediction error signal byexecuting inverse quantization and inverse orthogonal transform on thevalue quantized through the first transform process; executing a colorspace restoration process; the color space restoration process includingtransforming the subblock subjected to the orthogonal transform and thequantization in the YUV format by the first transform process into theRGB format; and executing an ACT map generation process, the ACT mapgeneration process including generating an ACT map including informationregarding a position of the subblock and the number of subblockstransformed from the YUV format to the RGB format by the color spacerestoration process, and wherein the determination process includesexecuting determining either color space format between the RGB formatand the YUV format to execute the intra prediction and the interprediction based on the ACT map.
 20. The non-transitorycomputer-readable storage medium according to claim 19, wherein thedetermination process includes determining either color space formatbetween the RGB format and the YUV format, in which the intra predictionand the inter prediction are executed on a current processing blockbased on the ACT map in a reference block at the same position as thecurrent processing block in a reference picture which is referred to inthe inter prediction.